LISTSERV at the University of Georgia
Menubar Imagemap
Home Browse Manage Request Manuals Register
Previous messageNext messagePrevious in topicNext in topicPrevious by same authorNext by same authorPrevious page (January 2006, week 5)Back to main SAS-L pageJoin or leave SAS-L (or change settings)ReplyPost a new messageSearchProportional fontNon-proportional font
Date:         Sun, 29 Jan 2006 07:46:56 -0500
Reply-To:     "Richard A. DeVenezia" <rdevenezia@WILDBLUE.NET>
Sender:       "SAS(r) Discussion" <SAS-L@LISTSERV.UGA.EDU>
From:         "Richard A. DeVenezia" <rdevenezia@WILDBLUE.NET>
Subject:      Re: What is maximum number of variables allowed
Comments: To:

David L Cassell wrote: > art297@NETSCAPE.NET replied: >> Then I was even more surprised that 2 million worked after having >> failed previously. >> >> Then, I was totally confused when SAS9 failed with 2097150 and >> returned the message that the maximum allowed for the file was >> 16777191. >> >> I would think that someone from SAS needs to respond regarding what >> the actual number is. > > I don't think there *is* a fixed number. > > I think that your work, as well as other experiments around here, are > demonstrating > that this is memory-dependent. Which could mean that - as your OS > adjusts the > amount of memory available to the SAS system - you may continue to get > fluctuating > results. > > Of course, I'm probably wrong. But it seems clear that the number of > variables > (and the length of the PDV created by that number of variables) is > going to be > memory-dependent. > > But larger than we usually work with... > > David


Interestingly 16,777,191 is 2**24.

A row with 2**24 numeric variables (double precision reals) of length 8 requires 2**27 bytes of storage/working space, plus overhead and alignment issues (if any)

One speculation is that internally 32-bit positive integers (31 if signed) are used to index into the memory buffer of a row, or at least index into a hash mapping to the address of a variable -- this would indicate the maximum allocation of a rows internal buffer is 4GB (2**32) or 2GB (2**31).

There is a 2**4 factor ratio between 2**31 and 2**27. This might indicate the internal scheme for dealing with a single wide row is perhaps utilizing a paging algorithm that requires adding 16 bytes to each variable to make it's value locatable in the working space.

Richard A. DeVenezia

Back to: Top of message | Previous page | Main SAS-L page