LISTSERV at the University of Georgia
Menubar Imagemap
Home Browse Manage Request Manuals Register
Previous messageNext messagePrevious in topicNext in topicPrevious by same authorNext by same authorPrevious page (January 2008, week 4)Back to main SAS-L pageJoin or leave SAS-L (or change settings)ReplyPost a new messageSearchProportional fontNon-proportional font
Date:         Mon, 28 Jan 2008 14:29:48 -0500
Reply-To:     LQLIU <lingqun@GMAIL.COM>
Sender:       "SAS(r) Discussion" <SAS-L@LISTSERV.UGA.EDU>
From:         LQLIU <lingqun@GMAIL.COM>
Subject:      Re: Efficiency with large datasets
Comments: To: Wensui Liu <liuwensui@gmail.com>
In-Reply-To:  <1115a2b00801281046h3bf12321jcf1527c762fb33dd@mail.gmail.com>
Content-Type: text/plain; charset=ISO-8859-1

Hi JB and Wensui,

I did something similar too. To speed the process, I first categorized variables into groups based on the variable types and the associated formats. For continuous variables, producing Xtable is meaningless and waste lots of time.

Hope this helps,

Lingqun

On Jan 28, 2008 1:46 PM, Wensui Liu <liuwensui@gmail.com> wrote: > Hi, JB, > I have a macro to do the same thing you want to do but can't show you here. > however, the idea is very simple. i am using a macro to loop through > each ind variable and using data step instead of proc freq to > calculate the percent and count. > for thousands of variables, it still takes at least a day to run on unix. > > > On Jan 28, 2008 1:35 PM, JB <john_banana@yahoo.com> wrote: > > I have a data set with a target variable and 45000 other variables. > > I wish to produce an Information Value for each variable. > > > > In order to cut the processing time down I have cut the number of records > > down to 2000 but each run through the data to produce a crosstab still takes > > ages. > > > > > > proc freq data = tran.var_deriv_stg3 noprint; > > tables &tvar1 * target > > /missing norow nocol nopercent nocum > > out = temp1(keep = &tvar1 target count); > > run; > > > > > > Is there any way I can produce these xtabs without the whole file being > > processed each time? > > > > Thanks > > > > > > -- > =============================== > WenSui Liu > Statistical Project Manager > ChoicePoint Precision Marketing > (http://spaces.msn.com/statcompute/blog) > =============================== >


Back to: Top of message | Previous page | Main SAS-L page