LISTSERV at the University of Georgia
Menubar Imagemap
Home Browse Manage Request Manuals Register
Previous messageNext messagePrevious in topicNext in topicPrevious by same authorNext by same authorPrevious page (October 2009, week 1)Back to main SAS-L pageJoin or leave SAS-L (or change settings)ReplyPost a new messageSearchProportional fontNon-proportional font
Date:         Mon, 5 Oct 2009 10:43:59 -0400
Reply-To:     Michael Raithel <michaelraithel@WESTAT.COM>
Sender:       "SAS(r) Discussion" <SAS-L@LISTSERV.UGA.EDU>
From:         Michael Raithel <michaelraithel@WESTAT.COM>
Subject:      Re: Merging Very Large Datasets
In-Reply-To:  <f481b97c-e7c3-4948-92be-57ef36ddba61@p10g2000prm.googlegroups.com>
Content-Type: text/plain; charset="us-ascii"

Dear SAS-L-ers,

Claus Yeh posted the following:

> > Dear SAS guru's, > > I have two very large SAS datasets. Each has 100000 variables and > 5000 observations. Is there an efficient way to merge these two > datasets without taking hours? > > thank you so much, > Claus, yeh, you could probably merge these SAS data sets, and Patrick has provided some good questions and a good source of information for you to consider. One question that I had--and this is purely rhetorical, so don't answer aloud--is: Do you really need to _MERGE_ the two SAS data sets, or would "concatenating" them meet your needs?

If the latter approach works, then consider using PROC APPEND, who is in my inner circle of SAS procedure friends. PROC APPEND will update the "BASE" SAS data set in place by Supergluing the other SAS data set to the bottom of it. There is a lot to know about PROC APPEND and how it may affect the Base SAS data set. Instead of me writing, and writing, and writing some more, I will just refer you to relevant page of the well-written SAS Procedures documentation:

http://support.sas.com/documentation/cdl/en/proc/61895/HTML/default/a000070934.htm

I would echo Alan's incredulity at the number of variables (100K) in your SAS data set! Does your local health department know that many SAS variables are crowded together in such cramped conditions?!?!?

Best of luck in all of your SAS endeavors!

I hope that this suggestion proves helpful now, and in the future!

Of course, all of these opinions and insights are my own, and do not reflect those of my organization or my associates. All SAS code and/or methodologies specified in this posting are for illustrative purposes only and no warranty is stated or implied as to their accuracy or applicability. People deciding to use information in this posting do so at their own risk.

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Michael A. Raithel "The man who wrote the book on performance" E-mail: MichaelRaithel@westat.com

Author: Tuning SAS Applications in the MVS Environment

Author: Tuning SAS Applications in the OS/390 and z/OS Environments, Second Edition http://www.sas.com/apps/pubscat/bookdetails.jsp?catid=1&pc=58172

Author: The Complete Guide to SAS Indexes http://www.sas.com/apps/pubscat/bookdetails.jsp?catid=1&pc=60409

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Size does matter. - Tagline for Godzilla, 1998 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++


Back to: Top of message | Previous page | Main SAS-L page