Date: Wed, 14 Jul 1999 15:39:51 -0400
Reply-To: cclark@URGRGCC.EDU
Sender: "SPSSX(r) Discussion" <SPSSX-L@LISTSERV.UGA.EDU>
From: cclark@URGRGCC.EDU
Subject: Best choice of score for validation study
Content-type: text/plain; charset=US-ASCII
Hello the list--
I am beginning to clean up placement test data in preparation for
validation studies for cutting scores, developmental program review,
etc. My experience with tests and measures is "long ago and far
away", so I could use some advice on several issues.
For the past five years, we have used equivalent versions of the
same tests for placement. Reading tests include Degrees of
Reading Power, forms T and U, and writing test include
Conventions of Written English, forms K and L. Each year, the test
form changes, alternating between T and U for reading, and K and
L for writing. The forms are slightly different, so the scaled scores
for one do not match those for the other version of the test. Once
scaled, the percentile scores are identical.
The cutting scores for Writing and Reading developmental courses
have been set at about the 50th percentile, with the split between
Writing I and Writing II set at about the 20th percentile.
We have approximately 400 new students each year with data for 5
years. Students are placed into 3 levels of writing courses (2
developmental, 1 college level) or into a developmental reading
course on the basis of the placement tests.
We have recorded raw scores, but scaled scores and normed
percentile scores can be estimated and manually entered on the
basis of equivalency tables provided by the test publisher. (No
equations or any other data from the norming group is available.)
Would it be appropriate to combine data from more than one year,
or should each year's analysis be run separately? Which type of
score should be used, given the difference in forms used.
As a criteria variable, student grades (A through F plus
withdraw/drop) are available, with C and above being considered
successful completion of a course.
Scaled and Percentiles tend to have gaps and repeated scores as
the raw score changed one unit. Part of the conversion charts
for the Conventions of Written English would be:
Raw Scaled K Scaled L Scaled Percentile
40 325 325 325= 95
39 324 325 324= 90
38 323 325 323= 84
37 323 324 322= 76
36 322 323 321= 67
35 321 322 320= 59
34 320 321 319= 50
33 319 319
32 319 50
Any suggestions on what type of score to use (raw, scaled,
percentile) and the type of analysis method to use to determine the
best cutting score would be appreciated.
Catherine S. Clark
Institutional Research
University of Rio Grande
Rio Grande OH 45674
(740) 245-7532 voice, 5035 fax
"I have yet to find the man, however exalted his station,
who did not do better work and put forth greater effort
under a spirit of approval than under a spirit of
criticism." - Charles Schwab
