Date:         Sun, 1 Jul 2007 20:59:33 +0000
Reply-To:     iw1junk@COMCAST.NET
Sender:       "SAS(r) Discussion" <SAS-L@LISTSERV.UGA.EDU>
From:         Ian Whitlock <iw1junk@COMCAST.NET>
Subject:      Re: Help! Accessing observations from a dataset using a lookup
              table (with constraint)
Comments: cc: Michael Raithel <michaelraithel@WESTAT.COM>

Summary: Read the problem
#iw-value=1

Michael,

You wrote in part:

   Howard, oh I gasped all right, my friend, I gasped BIG TIME! I guess
   that the noise of the traffic on the Beltway between Rockville and
   Washington, DC must have muted it out:-)

   Actually, _BECAUSE_ of the amazing size of the MASTER data set, I ruled
   out an in-memory solution such as Hash Tables or Formats. I didn't just
   choose a SAS Index solution simply because I wrote the book. If the
   prohibition against "modifying" MASTER applies even to building a SAS
   index on ID, then I believe Alon is left standing alone without a valid
   tool to use.

However, in the problem Alon <akadas@GMAIL.COM> explained that the big file
must be read sequentially, so an index solution is not available, and that
he needed only 300 records.  It is easy to store 300 ID's in memory, read
(for some time), checking each ID to see if it is in the wanted ID's.  So a
a partial in memory solution is most appropriate.  Since there are many
ID's to check a hash would be the one most likely to complete.

I wonder how reliable the data on the file is and what sort of information
it contains.  The answers may be worth a bigger gasp than the size.

Ian Whitlock