I'm pretty sure that the memory usage doesn't 'peg' because, SAS is
smart enough to know that the required operation can't fit in memory,
so it doesn't even try.
I don't know of any reason for the increase. So, I'd suggest rebuilding
the files, sorts, and indexed for V8. We have some monster files with
indexes, and the Index has continued to work well in our transition from
V6 from V8 (actually V8 using the V6 files).
In either case, even 90 minutes might be a tad excessive. I'm not
sure exactly what you are doing, but I'd suggest that it is quite
possible that creating a Vector and using it as the 'extract key'
could decrease the execution time to something around 15 minutes
for either V6 or V8.
In article Don Stanley <don_stanley@XTRA.CO.NZ> wrote:
> Wow, thats a monster of a difference.
> Was the indexed file created and indexed under V8 or is it a V6 file.
> real ideas about what is causing the problem but if that is the
> situation try rebuilding the index in V8 and see if any diff.
> I've often found in V6 that having the large indexed file physically
> sorted by the key is useful, as the index overheads seem lower -- I
> gather that the mechanism for reading the index (not the file, but the
> index) performs better when the file is also sorted.
> Overall tho, no real ideas, it just sounds like the I/O isn't
> performing, which is why I suggested the things above.
> Stephen Dunn wrote:
> > What is odd about the job on NT is that it doesn't seem to
> > want to use memory much at all. Looking at the NT task
> > manager while the job is running I see 95% of the 2Gb of
> > physical memory is free. Since this is an I/O bound
> > does setting memsize to anything other than its default 0
> > help at all? If memory is not being used much, why isn't
> > I was interested in your remark that the number of
> > matters. The number of columns presumably affects
> > between the file buffers and the program data vector - is
> > that CPU or I/O time?
> > The job in question has a data step that merges one file
> > 7 million rows and 200 columns with another of 400,000
> > and 150 columns, reading the larger file in indexed order
> > and the smaller in physical order. This takes c. 90
> > in V6 and 8 hours in V8.
> > Stephen Dunn
> > Highway Insurance Statistician
> > Tel (01277 266) 253
> > Don Stanley wrote:
> > How many fields are in the tables being read?
> > SAS requires much more space in V8 for
> > such as the program data vector, in order to
> > support large field names. A statement I have
> > from SAS tech support (relating to MVS) is to
> > expect to use a noticeable amount of extra
> > over v6.
> > I would be interested in knowing whether the V8
> > job under NT that prompted this discussion is
> > causing the NT swap file to go ballistic? If so,
> > it is a symptom of the job using more memory
> > before and needing to page in and out of memory
> > much more.
> > Don
> Don Stanley, B.SC, Dip O.R.S, MNZCS Director, Sysware Consulting
> Box 634, Wellington, NEW
> EMAIL:: email@example.com
> Author:: Beyond the obvious with SAS Screen Control Language.
> Author:: Solutions for your GUI Applications Development Using SAS/AF
> FRAME Technology
> Genealogy:: http://www.geocities.com/don_stanley_nz/family.htm
> SAS Tips:: http://www.geocities.com/don_stanley_nz/sastips.htm
Sent via Deja.com http://www.deja.com/
Before you buy.