LISTSERV at the University of Georgia
Menubar Imagemap
Home Browse Manage Request Manuals Register
Previous messageNext messagePrevious in topicNext in topicPrevious by same authorNext by same authorPrevious page (November 1998, week 2)Back to main SAS-L pageJoin or leave SAS-L (or change settings)ReplyPost a new messageSearchProportional fontNon-proportional font
Date:         Tue, 10 Nov 1998 15:14:18 -0500
Reply-To:     "Stetz, Cynthia (PCM - NJ)" <CStetz@NA2.US.ML.COM>
Sender:       "SAS(r) Discussion" <SAS-L@UGA.CC.UGA.EDU>
From:         "Stetz, Cynthia (PCM - NJ)" <CStetz@NA2.US.ML.COM>
Subject:      Re: Funky Stuff in SAS?
Comments: To: "Self, Karsten" <Karsten.Self@SCHWAB.COM>
Content-Type: text/plain; charset="iso-8859-1"

Karsten and all,

One way around the finite size of a UNIX filesystem when using the SAS large file support is to define a library across several physical filesystems (or directories). The catch here is that SAS will only put one partition of the large file in each directory, and the partsize is limited to the smallest available slice of space in any of the directories.

If the original poster of this problem would like to discuss this further, you may mail me directly at cynthia_stetz@ml.com <mailto:cynthia_stetz@ml.com> I'm sorry, but I deleted the original post.

Regards,

Cynthia

-----Original Message----- From: Self, Karsten [SMTP:Karsten.Self@SCHWAB.COM] Sent: Tuesday, November 10, 1998 2:50 PM To: SAS-L@UGA.CC.UGA.EDU Subject: Re: Funky Stuff in SAS?

I disagree with Bernard's diagnosis. Disk full results in a dialog or message "out of resources" or "insufficient space". Baba is running HPUX. SAS does not offer native large file (> 2GB) support on this platform, though a large file experimental mode is available. SASWORK could be 10 GB or 10 TB, he still can't produce a single system file of more than 2 GB.

The workaround is to enable SAS large file support. The confirmation is to watch file size and disk utilization within SAS WORK.

You typically can't allocate space in Unix as on MVS. A filesystem, once built, has a finite size, and is modifiable only by the root user. Users may have quotas or resource limits imposed (man 'quotas' and 'ulimit'), though this usually isn't the case. You *can* point to a larger space, if one is available, by specifying a different WORK system option at SAS startup.

2 GB is roughly:

1m records X 2k lrecl 2m records X 1k lrecl 4m records x 500 bytes lrecl

...a pattern should be emerging.

My usual question is whether or not the sort is necessary. It often isn't, or can be postponed, if you can do subsetting with a format, apply CLASS variables to a summary, or use an index. A standard practice is:

sort subset summarize

A much more efficient method is:

subset summarize (based on existing collation sequence) sort summarize

...tossing as much of the data as possible along the way and using views where possible. The NOTSORTED option can be used to allow summarization of data in which some grouping of like keys exists, but overall ordering isn't guaranteed.

I responded to an optimization problem last December on SAS-L which makes an interesting case study. Original problem, tactics, final code, and optimizations, are available in the following links.

http://x8.dejanews.com/getdoc.xp?AN=309293810 http://www.dejanews.com/getdoc.xp?AN=299366238

Karsten M. Self (Karsten.Self@schwab.com) Trilogy Consulting

What part of "gestalt" don't you understand?

WARNING: All e-mail sent to or from this address will be received by the Charles Schwab corporate e-mail system and is subject to archival and review by someone other than the recipient.

> ---------- > From: Ashiru, Babatunde[SMTP:AshiruB@WHITEOAKSEMI.COM] > Reply To: Ashiru, Babatunde > Sent: Tuesday, November 10, 1998 11:09 AM > To: SAS-L@UGA.CC.UGA.EDU > Subject: Re: Funky Stuff in SAS? > > Ok, points well taken but I have gotten as far as several hundred > thoudsands > over 1 million records in the past without any problem...albeit I > seldom get > I/O error message. We thought it was a tmp space problem; allocated > more > space and/or moved the process to another server with over 10 > gigabytes of > hard-disk space but all to no avail? > > BRA > > -----Original Message----- > From: Bernard Tremblay [SMTP:bernard@capitale.qc.ca] > Sent: Tuesday, November 10, 1998 2:05 PM > To: AshiruB@WHITEOAKSEMI.COM > Subject: Re: Funky Stuff in SAS? > > Hi, > > This funky stuff is called : DISK FULL ! > > You're out of space on the volume that is used for > sorting > (usualy the work library) or the volume where is the > 'PRDCNTDS' > library. > Try allocate more work space or find another volume for it... > > regards, > Bernard Tremblay >


Back to: Top of message | Previous page | Main SAS-L page