LISTSERV at the University of Georgia
Menubar Imagemap
Home Browse Manage Request Manuals Register
Previous messageNext messagePrevious in topicNext in topicPrevious by same authorNext by same authorPrevious page (September 2000, week 2)Back to main SAS-L pageJoin or leave SAS-L (or change settings)ReplyPost a new messageSearchProportional fontNon-proportional font
Date:   Mon, 11 Sep 2000 15:40:13 GMT
Reply-To:   amichiel@EARTHLING.NET
Sender:   "SAS(r) Discussion" <SAS-L@LISTSERV.UGA.EDU>
From:   amichiel@EARTHLING.NET
Organization:   Deja.com - Before you buy.
Subject:   Re: Disastrous performance in V8 cf V6

I'm pretty sure that the memory usage doesn't 'peg' because, SAS is smart enough to know that the required operation can't fit in memory, so it doesn't even try. I don't know of any reason for the increase. So, I'd suggest rebuilding the files, sorts, and indexed for V8. We have some monster files with indexes, and the Index has continued to work well in our transition from V6 from V8 (actually V8 using the V6 files). In either case, even 90 minutes might be a tad excessive. I'm not sure exactly what you are doing, but I'd suggest that it is quite possible that creating a Vector and using it as the 'extract key' could decrease the execution time to something around 15 minutes for either V6 or V8.

In article Don Stanley <don_stanley@XTRA.CO.NZ> wrote: > Wow, thats a monster of a difference. > Was the indexed file created and indexed under V8 or is it a V6 file. No > real ideas about what is causing the problem but if that is the > situation try rebuilding the index in V8 and see if any diff. > > I've often found in V6 that having the large indexed file physically > sorted by the key is useful, as the index overheads seem lower -- I > gather that the mechanism for reading the index (not the file, but the > index) performs better when the file is also sorted. > > Overall tho, no real ideas, it just sounds like the I/O isn't > performing, which is why I suggested the things above. > > Don > > Stephen Dunn wrote: > > > > > > > What is odd about the job on NT is that it doesn't seem to > > want to use memory much at all. Looking at the NT task > > manager while the job is running I see 95% of the 2Gb of > > physical memory is free. Since this is an I/O bound problem > > does setting memsize to anything other than its default 0 > > help at all? If memory is not being used much, why isn't it? > > > > I was interested in your remark that the number of variables > > matters. The number of columns presumably affects transfers > > between the file buffers and the program data vector - is > > that CPU or I/O time? > > > > The job in question has a data step that merges one file of > > 7 million rows and 200 columns with another of 400,000 rows > > and 150 columns, reading the larger file in indexed order > > and the smaller in physical order. This takes c. 90 minutes > > in V6 and 8 hours in V8. > > > > Stephen Dunn > > Highway Insurance Statistician > > Tel (01277 266) 253 > > > > Don Stanley wrote: > > How many fields are in the tables being read? > > SAS requires much more space in V8 for "overheads" > > such as the program data vector, in order to > > support large field names. A statement I have had > > from SAS tech support (relating to MVS) is to > > expect to use a noticeable amount of extra memory > > over v6. > > > > I would be interested in knowing whether the V8 > > job under NT that prompted this discussion is > > causing the NT swap file to go ballistic? If so, > > it is a symptom of the job using more memory than > > before and needing to page in and out of memory > > much more. > > > > Don > > > -- > Don Stanley, B.SC, Dip O.R.S, MNZCS Director, Sysware Consulting > Group > Box 634, Wellington, NEW ZEALAND > > http://www.sysware.co.nz > EMAIL:: don_stanley@xtra.co.nz > > Author:: Beyond the obvious with SAS Screen Control Language. > Author:: Solutions for your GUI Applications Development Using SAS/AF > FRAME Technology > > http://www.geocities.com/don_stanley_nz/don_home.htm > Genealogy:: http://www.geocities.com/don_stanley_nz/family.htm > SAS Tips:: http://www.geocities.com/don_stanley_nz/sastips.htm >

Sent via Deja.com http://www.deja.com/ Before you buy.


Back to: Top of message | Previous page | Main SAS-L page