LISTSERV at the University of Georgia
Menubar Imagemap
Home Browse Manage Request Manuals Register
Previous messageNext messagePrevious in topicNext in topicPrevious by same authorNext by same authorPrevious page (October 2005, week 2)Back to main SAS-L pageJoin or leave SAS-L (or change settings)ReplyPost a new messageSearchProportional fontNon-proportional font
Date:         Wed, 12 Oct 2005 21:51:13 -0700
Reply-To:     David L Cassell <davidlcassell@MSN.COM>
Sender:       "SAS(r) Discussion" <SAS-L@LISTSERV.UGA.EDU>
From:         David L Cassell <davidlcassell@MSN.COM>
Subject:      Re: sas Performance Enhancement
In-Reply-To:  <200510121602.j9CF6VJ3013766@malibu.cc.uga.edu>
Content-Type: text/plain; format=flowed

paul.dorfman@FCSO.COM expertly replied: >Since its inception, SASFILE has been benchmarked at least by one person I >know of. Originally (and understandably) I was highly enthused when >SASFILE appeared in V8, but minimal testing quickly showed that, with >notable exceptions (see below) it has been a sort of disappointment, >mainly because, following the common wisdom of "memory is 100 faster than >disk", I had anticipated much more sizeable performance improvements. > >[really useful documentation of times elided by a lazy slob] > >Not to say that the prebuffered file yields no improvement in reading >performance, but in real time, it is definitely not 3 or even 2 orders of >magnitude. Heck, not even 1. > >The most sizeable improvement observed in the speed of direct reads is >undoubtedly owing to the fact that with SASFILE, all file pages are >already preloaded to the buffer. Without SASFILE, if an observation is >requested and it is not in the currently buffered page, it must be >unbuffered, and the page containing the observation must be buffered in, >while when the file is prebuffered, this obviously is not necessary and >does not happen. The observed performance differences are mainly owing to >the fact that same pages get buffered repeatedly. The proof is in the >sequential read, where each page is buffered only once, and hence the >overall speed difference is negligible. By the same token, if pages are >read in order, for instance, as > > do key = 1 to n by 10 ; > set halfgig key = key nobs = n ; > end ; > do ptr = 1 to n by 10 ; > set halfgig point = ptr nobs = n ; > end ; > >the SASFILE performance improvements related to the indexed/random reads >dwindles to the almost why-bother level.

Dale McLerran and I have found (Dale deserves most of the credit here) that SASFILE makes a big difference in time when using PROC SURVEYSELECT to do bootstrapping. That is:

sasfile targetpopulationfile open;

proc surveyselect data=targetpopulationfile out=bootstrapfile method=urs seed=4954734 outhits reps=1000; run;

It appears that the proc doesn't cache the data set beforehand, so this process saves a lot of I/O time.

So I think that the use of SASFILE needs to be restrained. It doesn't solve all problems, and it can cause headaches if the file is too big. But some cases, like your last example and the above, show that there can be some merit in its use.

David -- David L. Cassell mathematical statistician Design Pathways 3115 NW Norwood Pl. Corvallis OR 97330

_________________________________________________________________ Express yourself instantly with MSN Messenger! Download today - it's FREE! http://messenger.msn.click-url.com/go/onm00200471ave/direct/01/


Back to: Top of message | Previous page | Main SAS-L page