Date:         Wed, 18 Dec 1996 22:16:13 GMT
Reply-To:     jk93 <jk93@EROLS.COM>
Sender:       "SAS(r) Discussion" <SAS-L@UGA.CC.UGA.EDU>
From:         jk93 <jk93@EROLS.COM>
Organization: no fixed address
Subject:      Re: Sorting huge datasets: SAS 6.08 under wfwg.
Content-Type: Text/Plain; charset=US-ASCII

In article <96352.151515KLASSEN@UVVM.UVic.CA>, KLASSEN@UVVM.UVIC.CA says...
>
>jk93 <jk93@EROLS.COM> writes:
>>I'm using SAS 6.08 (wfwg3.11). I'm sorting huge datasets (almost 2 million
>>cases, 30 variables), though only sorting on one variable (like ID), using
>>Proc Sort. But it seems that SAS makes NO use at all of extended memory
when
>>sorting. I have 32 meg (with an 8 meg vcache), but the disk is just
>>constantly crunching (heavily) during the sort. (I'm wondering, also, how
bad
>>this is for the disk, during these hour-long sorts). But even with much
>>smaller datasets, the disk acts the same way, with seemingly no use of
>>memory. Any ideas?
>
>Buy more RAM.  How much?
>
>Assuming you have 8 bytes per variable, and 30 variables, and 2 million
cases,
>and you are sorting on 1 variable, you will need at least:
>
>     (30*8 + 1*8) * 2 million
>
>i.e., 496 million bytes of RAM, to store your file.
>
>About 500MB of RAM will only cost about $5,000. -- assuming that you have
>enough empty SIMM-slots on the computer's motherboard to hold the RAM.
>
 
 
 
 As i said, even with much smaller datasets,
the sorting seems to be done mainly on disk.  To be even more clear,
no matter what the size of
the data set, it makes no noticeable use of memory.
 
I would think that even with huge datasets, it could at least put SOME of
the data in memory to do the sorting.
 
On the other hand, I also use SYSTAT, which does the sorting in memory (when
it can fit), and it's faster (with little disk activity)..