LISTSERV at the University of Georgia
Menubar Imagemap
Home Browse Manage Request Manuals Register
Previous messageNext messagePrevious in topicNext in topicPrevious by same authorNext by same authorPrevious page (September 2000, week 2)Back to main SAS-L pageJoin or leave SAS-L (or change settings)ReplyPost a new messageSearchProportional fontNon-proportional font
Date:   Sat, 9 Sep 2000 23:42:30 GMT
Reply-To:   Aswin <A.Bouwmeester@INTER.NL.NET>
Sender:   "SAS(r) Discussion" <SAS-L@LISTSERV.UGA.EDU>
From:   Aswin <A.Bouwmeester@INTER.NL.NET>
Organization:   UUNET-NL (http://www.nl.uu.net)
Subject:   Re: Disastrous performance in V8 cf V6

This problem is not memory related as some suggest but I/O related. I do not know the cause, but this is how it works: SAS datasets are paged, each page can contain multiple obs. When reading data from disk it is done page by page, not obs by obs. So if you want one obs SAS reads a whole page. If you want another one that is in the same page, there won't be any I/O. If you want a second that's not on the same page, there is I/O. If you use an index to access the data it is not very likely that subsequent obs are an the same page, most obs require a new page to be read. In the end pages are read multiple times, could be as often as there are obs in a page. If the data was sorted you would be sure that each page will be read only once. Conclusion: Indexed data access can result in suboptimal I/O.

Some hints -Pagesize can be set using bufsize option -Pagesize and number of observation in a page can be queried using proc contents

Aswin

On 31 Aug 00 14:09:00 GMT, Stephen_D@HWAY.CO.UK (Stephen Dunn) wrote:

>This message is in MIME format. Since your mail reader does not understand >this format, some or all of this message may not be legible. > >------_=_NextPart_001_01C01355.02C7E8F0 >Content-Type: text/plain; > charset="iso-8859-1" > > We run a middling size SAS database (one large table >of 10million rows, a few others of 100,000s rows) on an NT server which we >have just converted to using V8 software and V8 table formats from V6. The >database updates are now taking much longer than before and its performance >on queries is much worse. The difficulty seems to be around accessing files >in index order rather than physical order - in some cases this is taking >almost ten times as long as V6 used to on the same data in V6 formats. Even >the simple act of reading a file using an index as BY variables after a SET >statement is painfully slow. > > Has anyone had similar experiences? If so, is it curable? > > Stephen Dunn > Highway Insurance Statistician > Tel (01277 266) 253 > > >------_=_NextPart_001_01C01355.02C7E8F0 >Content-Type: text/html; > charset="iso-8859-1" >Content-Transfer-Encoding: quoted-printable > ><!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN"> ><HTML> ><HEAD> ><META HTTP-EQUIV=3D"Content-Type" CONTENT=3D"text/html; = >charset=3Diso-8859-1"> ><META NAME=3D"Generator" CONTENT=3D"MS Exchange Server version = >5.5.2650.12"> ><TITLE>Disastrous performance in V8 cf V6</TITLE> ></HEAD> ><BODY BGCOLOR=3D"#F8F8F8"> ><UL><UL> ><P><I>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</I> <FONT SIZE=3D4>We = >run a middling size SAS database (one large table of 10million rows, a = >few others of 100,000s rows) on an NT server which we have just = >converted to using V8 software and V8 table formats from V6. The = >database updates are now taking much longer than before and its</FONT> = ><FONT SIZE=3D4>performance</FONT> <FONT SIZE=3D4>on queries is much = >worse. The difficulty seems to be around accessing files in index order = >rather than physical order - in some cases this is taking almost ten = >times as long as V6 used to on the same data in V6 formats. Even the = >simple act of reading a file using an index as BY variables after a SET = >statement is painfully slow.</FONT></P> > ><P><FONT SIZE=3D4>Has anyone had similar experiences? If so, is it = >curable?</FONT> ></P> > ><P><FONT SIZE=3D4>Stephen Dunn</FONT> ><BR><FONT SIZE=3D4>Highway Insurance Statistician</FONT> ><BR><FONT SIZE=3D4>Tel&nbsp; (01277 266) 253</FONT> ></P> ></UL></UL> ></BODY> ></HTML> >------_=_NextPart_001_01C01355.02C7E8F0--


Back to: Top of message | Previous page | Main SAS-L page