Date: Wed, 11 Nov 2009 15:12:03 +0100
Reply-To: Fernández Rodríguez, Dani
<DFernandez@CST.CAT>
Sender: "SAS(r) Discussion" <SAS-L@LISTSERV.UGA.EDU>
From: Fernández Rodríguez, Dani
<DFernandez@CST.CAT>
Subject: Re: macro problem for reading,
creating and merging several datasets
In-Reply-To: A<037AB3FF38D44C4BAFB5DFF3D06B57BA1F864B2A@EX-CMS01.westat.com>
Content-Type: text/plain; charset="iso-8859-1"
Thank you Mike for your 'View' point of view. I was using it last week when I was manipulating a 12.000.000 records file...
What I make me angry is that SAS is creating more new procedures..
PROC WHATEVER ... I never imagined it would appear a new one with this name
LOL !
Just kidding...
Well, I said 'hello' to Las Ramblas last day for you...
Daniel Fernandez
Barcelona.
-----Mensaje original-----
De: SAS(r) Discussion [mailto:SAS-L@LISTSERV.UGA.EDU] En nombre de Michael Raithel
Enviado el: miércoles, 11 de noviembre de 2009 13:54
Para: SAS-L@LISTSERV.UGA.EDU
Asunto: Re: macro problem for reading, creating and merging several datasets
Dear SAS-L-ers,
Cornel Lencar posted the following in this continuing discussion with Joe and Mark:
>
> Apparently I still have to work with the small files one by one and
> then aggregate the results into a final file, due to considerations of size
> of files and space on the server
>
Cornel, watching this discussion from the sidelines, I noted your statement: "due to considerations of size of files and space on the server". Depending upon your definition of "aggregate the results into a final file", you might be able to construct the final file by using a DATA Step View or an SQL View that effectively "concatenates" all of the small files into one big file without taking up very much real estate on your disks. SAS Views are essentially very small files that describe other files. SAS constructs the larger files, ad-hoc, when executing the view.
So, you could code something like:
data cornlib.lenclarge/view=cornlib.lenclarge;
set lencsmall1 lencsmall2 lencsmall3 lencsmall4 lencsmall5 lencsmall6 ...etc...;
run;
proc summary nway data=conrlib.lenclarge;
etc.
The data step would build a SAS view file and store it as a permanent file in your CORNLIB SAS data library. Thereafter your PROC SUMMARY or PROC WHATEVER would exploit that view; dynamically concatenating your small SAS data sets together and channeling all of their obs into the relevant SAS procedure.
So, what do you think? <--Purely rhetorical device; you don't have to answer... except it would be nice to know if you use this and it works:-)
Cornel, best of luck in all of your SAS endeavors!
I hope that this suggestion proves helpful now, and in the future!
Of course, all of these opinions and insights are my own, and do not reflect those of my organization or my associates. All SAS code and/or methodologies specified in this posting are for illustrative purposes only and no warranty is stated or implied as to their accuracy or applicability. People deciding to use information in this posting do so at their own risk.
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Michael A. Raithel
"The man who wrote the book on performance"
E-mail: MichaelRaithel@westat.com
Author: Tuning SAS Applications in the MVS Environment
Author: Tuning SAS Applications in the OS/390 and z/OS Environments, Second Edition
http://www.sas.com/apps/pubscat/bookdetails.jsp?catid=1&pc=58172
Author: The Complete Guide to SAS Indexes
http://www.sas.com/apps/pubscat/bookdetails.jsp?catid=1&pc=60409
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
However beautiful the strategy, you should occasionally look
at the results. - Winston Churchill
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
