Date: Tue, 17 Mar 2009 10:27:23 +0000
Reply-To: Ruben van den Berg <ruben_van_den_berg@hotmail.com>
Sender: "SPSSX(r) Discussion" <SPSSX-L@LISTSERV.UGA.EDU>
From: Ruben van den Berg <ruben_van_den_berg@hotmail.com>
Subject: Re: ceiling effect problem
In-Reply-To: <6.2.1.2.2.20090316101449.0a203628@hawaii.edu>
Content-Type: multipart/alternative;
Dear Ron and others,
This problem is familiar to me as well. What seems to work reasonably for us, is to use (what we call) 'skewed' scales. If a sample tends to be overly positive, we use a scale with more positive answers than negative ones, like:
1 (Very) bad
2 Neutral
3 Good
4 Very good
5 Excellent
We typically -though not always- observe that this yields univariate distributions that don't deviate too much from being normal or at least symmetrical.
However, when we conduct CSSs relating to governmental institutions, we have to reverse the skewed scale (an extra negative category) in order to achieve desirable distributions. It seems there's a lot of 'unhappy campers' when it comes to our government!
Well, it's possibly too late for you to adapt your rating scales anyway but I just wanted to point this out as an option.
Kind regards,
Ruben v.d. Berg
Date: Mon, 16 Mar 2009 10:29:20 -1000
From: schacht@hawaii.edu
Subject: Re: ceiling effect problem
To: SPSSX-L@LISTSERV.UGA.EDU
At 08:54 AM 3/16/2009, Roy Money wrote:
...I have longitudinal survey data on investigator ratings of achievement
in various goals such as
'commitment to interdisciplinary collaboration'
'exposure to divergent points of view'
'unexpected findings have served as source of new ideas',
as well as various ratings of satisfaction .
The rating scale is 1=Not at all, 4= Moderate and 7=Very Much.
As you might imagine we have a ceiling effect on some items and are
trying to work out a meaningful way to handle this
(the ceiling effect, % of responses that are coded at the maximum value,
ranges as high as 20-40% on some items)
We are considering using new anchors for the scale such as 5 = Very
Much and 7=Maximum Possible
but of course we still have the existing data which would need to be transformed.
Any suggestions or references on an appropriate way to rescale the
maximum values down or otherwise deal with this situation?
What you call "ceiling effects" is very common in satisfaction surveys. I've been working on a satisfaction survey redesign for more than a year, so this is an issue of some interest to us as well. Unfortunately, you are in a longitudinal survey situation, so you cannot so easily change the wording of the questions-- or even the wording of the answers. Any changes at all would affect your longitudinal analyses, wouldn't it?
In many satisfaction surveys, it seems that people are either happy campers, or unhappy campers. The happy campers tend to look for the highest satisfaction category on every question, and the unhappy campers look for the highest dissatisfaction category on every question. Sampling bias usually results in a higher probability of participation by happy campers unless there is some mechanism for forcing participation in a representative sample. The result is usually a J-shaped bimodal distribution, with the short leg at the "very dissatisfied" end of the scale, and the long leg at the "very satisfied" end of the scale, rather than a normal distribution.
If all of your questions result in distributions that are skewed positively (your "ceiling effect"), you might want to examine your sampling procedures, and what you can do to improve them.
Bob Schacht
Robert M. Schacht, Ph.D., Research Director
Pacific Basin Research and Training Center
1268 Young Street, Suite #204
Research Center, University of Hawaii
Honolulu, HI 96814
E-mail <schacht@hawaii.edu>
Phone 808-592-5904, FAX 808-592-5909
_________________________________________________________________
See all the ways you can stay connected to friends and family
http://www.microsoft.com/windows/windowslive/default.aspx
[text/html]
| 
