LISTSERV at the University of Georgia
Menubar Imagemap
Home Browse Manage Request Manuals Register
Previous messageNext messagePrevious in topicNext in topicPrevious by same authorNext by same authorPrevious page (May 2001, week 2)Back to main SAS-L pageJoin or leave SAS-L (or change settings)ReplyPost a new messageSearchProportional fontNon-proportional font
Date:         Tue, 15 May 2001 03:28:47 -0000
Reply-To:     sashole@bellsouth.net
Sender:       "SAS(r) Discussion" <SAS-L@LISTSERV.UGA.EDU>
From:         Paul Dorfman <paul_dorfman@HOTMAIL.COM>
Subject:      Re: out of memory error while processing large data
Comments: To: ghellrieg@T-ONLINE.DE
Content-Type: text/plain; format=flowed

>From: Gerhard Hellriegel <ghellrieg@T-ONLINE.DE> > >YES!! A lot of problems with memory and SAS V8!! We run it under >OS/390 >and we need up to 5 times more memory as in V6!

Gerhard,

Hmmmm... I thought I've tested many memory-demanding features of V8.1 under OS/390 and W2000P pretty much beyond the limit intended by the software and did not find it especially troublesome. I have even reported that V8's handle on memory usage with character temp arrays has become better than in V6 by a sizeable measure. 5 times? Wow! I would like to see an instance of that happening... when it matters. I mean, yes,

data _null_; run;

takes about twice as much memory to run in V8 than in V6, but due to the brevity of execution, the amount of data processed, and that of total amount of memory annihilated (< 10 MB) it did not seem overly disturbing.

>Especially with procs like FREQ, SUMMARY, TABULATE, ... Also SQL seems >to >have some problems.

Again, I have not seen many more problems with these in V8 than in V6, but maybe I just did not twist them hard enough. Normally, when CLASS procs start running out of memory, it is a fair indication that the cardinality of the CLASS variable(s) is too high to be handled by a binary tree, and other approaches (SORTing/BYing, hashing, etc.) might be knocking on the door.

>I think most of the statistical procs which work with a kind of tables > >(clusters, memory blocks, whatever) to keep some data in memory will > >also have problems.

That may very well be true; hard to say without running one. David and/or Dale would almost certainly report their grievances.

>We have one application with a PROC FREQ which runs in V6.09 with >about >80M region size and failed in V8 with 400M!

There is a good reason for it, which does not mean, however, that in V8, FREQ uses much more than in V6. In V6 under MVS, SAS handles memory quite differently from V8. In particular, the REGION specification in V6 is irrelevant with MEMSIZE=0 because SAS has a design feature to circumvent the safeguard imposed by REGION and grab as much memory as it needs. Not surprisingly, one would not find lots of love for that V6 feature in the humble camp of MVS system programmers. (They *have* found a way to have Big Brother cut the job off if it becomes too memory greedy.) In V8, SI have changed that behavior, and MEMSIZE=0 no longer can go to the sky, but only up to REGION specification. It seems more logical, and should help alleviate some understandable animosity between professional SAS programmers and their friends from the Central Command Unit. All the more that as far as I can judge by having run the same jobs using V6 and V8 in the same environment, the dreaded message "NO MKLEs FOUND" is gone for good, without the necessity to apply a V6 zap.

>We got it to run with 500M!

Sure, but again, for the reasons cited, it means *not* that in V6 and V8, the same step actually used 80MB and 500MB correspondigly. The log might be able to attest to it quite eloquently.

>You can refer to the SAS website for a big list with memory problems. >Some >of them should be fixed with 8.2, some in V9 (???)

Hard to argue with that; but how many of them have happened on the real computer?

Kind regards, ======================== Paul M. Dorfman Jacksonville, Fl ========================

_________________________________________________________________ Get your FREE download of MSN Explorer at http://explorer.msn.com


Back to: Top of message | Previous page | Main SAS-L page