Date: Mon, 24 Jan 2005 11:15:52 -0800
Sender: "SAS(r) Discussion" <SAS-L@LISTSERV.UGA.EDU>
From: "David L. Cassell" <cassell.david@EPAMAIL.EPA.GOV>
Subject: Re: Proc CORR running out of memory
Content-type: text/plain; charset=US-ASCII
"Chaudhury, Jayati [IT]" <jayati.chaudhury@CITIGROUP.COM> wrote:
> I am getting an out of memory error when I try to run proc corr on a
> with 784 observations and 6122 variables. The code is provided below.
> Is there a limitation on the size that Proc CORR can handle? I am
> using SAS 9.1 on Linux.
> This is urgent. So a prompt response will be greatly appreciated.
 This is a mailing list/newsgroup of SAS users who have jobs and
deadlines themselves. I'm sure you didn't mean to sound so rude as
to demand a 'prompt response' when we all have jobs just as
as you do.
 The PROC CORR documentation has specifics on the memory usage given
the number of variables, etc. It would have been much faster for you
to check that yourself. If you don't have a copy locally, you can read
the docs at sas.com .
 6122 variables is ridiculous. How can you possibly work with a
6122x6122 matrix of correlations? That's 18,736,381 unique
just considering the upper triangular matrix and ignoring all the main
diagonal. If you want to pick out 'large' correaltions, you'll get
so many spuriously significant results that you won't be able to
the valuable from the random. With that many correlations to wade
under the null hypothesis you would *expect* to find over 900,000 false
 With 6122 variables, you cannot manage when you have only 784
What do you expect to achieve with this sort of approach? Perhaps you
re-think your business process here.
David Cassell, CSC
Senior computing specialist