Date: Fri, 18 May 2007 14:41:13 -0400
Reply-To: Sigurd Hermansen <HERMANS1@WESTAT.COM>
Sender: "SAS(r) Discussion" <SAS-L@LISTSERV.UGA.EDU>
From: Sigurd Hermansen <HERMANS1@WESTAT.COM>
Subject: Re: Overprediction in logistic regression
Content-Type: text/plain; charset="us-ascii"
Good idea ...
The wide range of scores in the top 'bin' suggests that one might find
very different proportions of 'bad' across the distribution of scores.
From: firstname.lastname@example.org [mailto:email@example.com]
On Behalf Of Luo, Peter
Sent: Friday, May 18, 2007 10:12 AM
To: Alok; SAS-L@listserv.uga.edu
Subject: RE: Overprediction in logistic regression
See how predicted score distributed in the top bin.
From: SAS(r) Discussion [mailto:SAS-L@LISTSERV.UGA.EDU] On Behalf Of
Sent: Friday, May 18, 2007 2:15 AM
Subject: Overprediction in logistic regression
I am facing a very peculiar problem with the results of logistic
regression. Once i group the entire population on basis of their
predicted scores and try to compare the actual default rate and the
predicted average score for each bin, i observe that the model is
overpredicting at both ends of the score range, ie both at lower scores
as well as at higher scores. A sample distribution is like
Default Bucket Avg Score Avg bad rate
0<=Score<=0.0025 0.16% 0.05%
0.0025<Score<=0.0075 0.49% 0.33%
0.0075<Score<=0.0150 1.09% 0.93%
0.0150<Score<=0.0250 1.96% 1.80%
0.0250<Score<=0.0450 3.39% 3.50%
0.0450<Score<=0.0700 5.57% 6.03%
0.0700<Score<=0.1000 8.23% 9.61%
0.1000<Score<=0.1500 11.83% 13.85%
0.1500<Score<=0.2500 18.56% 17.19%
0.2500<Score<=1.0000 42.51% 21.23%
Total 2.78% 2.76%
The population is sizable in all bins. Its not concentrated in a
particular score range.
Here as seen the avg score exceeds the average bad rate at both ends.
What I expected was that it should be overpredicting at the lower score
ranges and underpredicting at the higher end.
Can someone please throw some light on what can be the possible reason?
I cannot change the bin boundaries since they are defined by business.
This message is the property of Draftfcb and contains information which
may be privileged or confidential. It is meant only for the intended
recipients and/or their authorized agents. If you believe you have
received this message in error, please notify us immediately by return
e-mail and destroy any printed or electronic copies of the message. Any
unauthorized use, dissemination, disclosure, or copying of this message
or the information contained in it, is strictly prohibited and may be
unlawful. Thank you for your cooperation.