Date: Fri, 24 Oct 2003 01:14:21 +0100
Reply-To: John Whittington <John.W@MEDISCIENCE.CO.UK>
Sender: "SAS(r) Discussion" <SAS-L@LISTSERV.UGA.EDU>
From: John Whittington <John.W@MEDISCIENCE.CO.UK>
Subject: Re: chi square analysis to identify the outliers
In-Reply-To: <E1ACpEK-00044M-00@coumxnn02.netbenefit.co.uk>
Content-Type: text/plain; charset="us-ascii"; format=flowed
At 16:50 23/10/03 -0700, David L. Cassell wrote (in small part):
>I don't think that you can reasonably rule out your high value when you
>can't really assume normality in your data.
Statistically speaking, that's obviously correct. However, as I wrote to
Karriere yesterday, if one of the purposes of the exercise is to select
'high-fliers' for prizes, and given that the mark has an upper bound of
100, if one didn't give a prize to someone who got 99/100, who on earth
WOULD get prizes. Indeed, you could equally have made the same comment had
the top mark obtained been 100. The bottom line is surely that if some
people are going to get marks so close to (or even equal to) the upper
bound, then there is no way that any method of trying to identify
'outliers' (however defined) is going to be useful for selecting prizewinners.
If the exam subject and/or marking scheme were such that the vast majority
of students' marks were well away from the bounds of the mark range, then,
with a reasonable number of subjects, an 'outlier detection' approach
might, of course, be much more reasonable for selecting those deserving of
'prizes and penalties' - although (unless one is really trying to identify
only the truely remakable - good or bad) it would probably be better to
give 'prizes and rewards' to those in defined regions of the tails of
distributions, even if they are not really definable as 'outliers'.
Indeed, such an approach would give sensible answers even with Karriere's
sample of 10, for which the correctly-calculated 95% CI is something like
24-92. If one makes all the necessary assumptions and uses that as the
criteria for 'prizes and penalties', then one would decide that just the 8
score deserved a penalty, and that the 99 (and perhaps also 92) one
deserved a prize - precisely what I suggested yesterday (and I imagine you
would probably also have suggested) on the basis of just eyeballing the
data and 'common sense'.
>I would certainly say that 8 out of 100 is a bad grade. But it doesn't
>look like an outlier given the rest of the data. Sorry.
Indeed; that really echoes what I wrote yesterday. In fact, 8 is not much
further from the mean (about -50) than is 99 above the mean (about
+41). Mind you, statistics aside, if I was in the business of handing out
'penalties', I think the student who got only 8/100 in an exam in which
half the students got 75/100 or more would be a prime target; indeed, in
the 'bad old days', it might well have been a 'penalty' that would have
impaired their ability to sit down for a while :-)
Kind Regards
John
----------------------------------------------------------------
Dr John Whittington, Voice: +44 (0) 1296 730225
Mediscience Services Fax: +44 (0) 1296 738893
Twyford Manor, Twyford, E-mail: John.W@mediscience.co.uk
Buckingham MK18 4EL, UK mediscience@compuserve.com
----------------------------------------------------------------
| 
