LISTSERV at the University of Georgia
Menubar Imagemap
Home Browse Manage Request Manuals Register
Previous messageNext messagePrevious in topicNext in topicPrevious by same authorNext by same authorPrevious page (June 2004, week 4)Back to main SAS-L pageJoin or leave SAS-L (or change settings)ReplyPost a new messageSearchProportional fontNon-proportional font
Date:         Tue, 22 Jun 2004 10:35:52 CEST
Reply-To:     datametric@CLUB-INTERNET.FR
Sender:       "SAS(r) Discussion" <SAS-L@LISTSERV.UGA.EDU>
From:         SUBSCRIBE SAS-L Stephane <datametric@CLUB-INTERNET.FR>
Subject:      Re: read a large flat file
Comments: To: radevenz@IX.NETCOM.COM
Content-Type: text/html; charset=ISO-8859-1

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> <HTML> <HEAD><TITLE></TITLE> <STYLE> body, table, tr, td, p {font-family: Verdana;font-size:12px;margin: 0px 0px 0px 0px} .bgtabl {BACKGROUND-REPEAT: no-repeat} </STYLE> </HEAD> <BODY bgProperties="fixed" bgcolor="#FFFFFF" background=""> Good morning,&nbsp;<BR> <BR>I take advantage of the situation to put an another question.&nbsp;<BR> <BR>I was developping this part of the program when the IT staff proposed to me to send me the data through a pointer instead of sending me a flat file. The application is based on a UNIX server and I receive datas onto a PC.&nbsp;<BR> <BR>I sometimes use the interface between C/dll and SAS so I think it&#8217;s a good idea because the server application hasn&#8217;t to write a flat file somewhere but : <BR>* I noted that the peek function support a length of 32k and, as you can see, I usually receive very big file. <BR>* the place into memory for this kind of pointer will be very important and will decrease performances.&nbsp;<BR> <BR>Does the method is profitable ?&nbsp;<BR> <BR>What is the method to receive a pointer with a size of 32+k ? ? ?&nbsp;<BR> <BR>Stéphane. <BR><BR><BR><BR>----Message d'origine---- <BR>&gt;Date: Mon, 21 Jun 2004 17:28:00 -0400 <BR>&gt;De: "Richard A. DeVenezia" <RADEVENZ@IX.NETCOM. COM><BR>&gt;Sujet: Re: read a large flat file <BR>&gt;A: SAS-L@LISTSERV.UGA.EDU <BR>&gt; <BR>&gt;Datametric wrote: <BR>&gt;&gt; Yes it's OK. <BR>&gt;&gt; <BR>&gt;&gt; In reality, this my first step and the second concern the managment <BR>&gt;&gt; of the length of the variables and the special characters into. <BR>&gt;&gt; Even if I use the validvarname = ANY, I can't read my first line with <BR>&gt;&gt; your code because it contains dot separators into the variable names <BR>&gt;&gt; and some columns width is greater than 32 : <BR>&gt;&gt; <BR>&gt;"datesMB.M.A.A10.X.1.Z5.0000.Z01.M.E.B.X*MB.M.A.L50.A.1.Z5.0000.Z06.M.E.B.X* <BR>&gt;&gt; <BR>&gt;MB.M.A.L50.A.1.Z5.0000.Z01.M.E.B.X*MB.M.A.A20.A.1.U4.0000.Z01.M.E.B.X*MB.M.A <BR>&gt;&gt; <BR>&gt;..A20.A.1.U5.1000.Z01.M.E.B.X*MB.M.A.L60.X.1.Z5.0000.Z01.M.E.B.X*MB.M.A.L62.X <BR>&gt;&gt; <BR>&gt;..1.Z5.0000.Z01.M.E.B.X*MB.M.A.A20.A.1.U5.2000.Z01.M.E.B.X*MB.M.A.A20.A.1.U5. <BR>&gt;&gt; <BR>&gt;2100.EUR.M.E.B.X*MB.M.A.A20.A.1.U5.2100.Z01.M.E.B.X*MB.M.A.A20.A.1.U5.2200.E <BR>&gt;&gt; <BR>&gt;UR.M.E.B.X*MB.M.A.A20.A.1.U5.2200.Z01.M.E.B.X*MB.M.A.A20.A.1.U6.1000.Z01.M.E <BR>&gt;&gt; .B.X*MB.M.A.L70.X.1.Z5.0000.Z01.M.E.B.X and so on" . <BR>&gt;&gt; <BR>&gt;&gt; But you didn't know because I didn't write because I was be able to <BR>&gt;&gt; manage them. So I will use your step in other cases (I have it !!) <BR>&gt;&gt; and load the names into macro-variables or into a table to manage <BR>&gt;&gt; them, in this case. Otherwise, if you have a solution.... <BR>&gt;&gt; <BR>&gt; &gt; Anyway, Thanck you for this profitable lesson. <BR>&gt;&gt; <BR>&gt;&gt; <BR>&gt;&gt; Kind regards, <BR>&gt;&gt; <BR>&gt;&gt; Stéphane. <BR>&gt; <BR>&gt;Take Peters excellent example and adapt it. <BR>&gt;If the column names are know to exceed 32 characters, than you will have to <BR>&gt;1. use a array name (serie1...serieN) <BR>&gt;or <BR>&gt;2. map the &gt;32 char name to &lt;=32 char name (compress out everything that is <BR>&gt;not a-z_0-9 as input to a truncatable sequence generator) <BR>&gt; <BR>&gt;Consider 1. The code has to generate <BR>&gt; <BR>&gt;attriblabel <BR>&gt;.... <BR>&gt; <SERIES-NAME><X>length=$<SOME <br Xth of length expected determination>&gt;variable in series&gt; label="<XTH line first from value delimited>" <BR>&gt;; <BR>&gt; <BR>&gt;input <BR>&gt; <SERIES-NAME>1 <BR>&gt;.... <BR>&gt; <SERIES-NAME><N><BR>&gt;; <BR>&gt; <BR>&gt;You will need to multipass the infile if you want to measure the data before <BR>&gt;the final data step that inputs the data into a SAS table. <BR>&gt;If you are confident the first <X>(you pick X) rows are a good sample <BR>&gt;containing the maximum length of data values to be encountered throughout <BR>&gt;the entire data file, then you will only have to measure the data based on <BR>&gt;the first X data rows. <BR>&gt; <BR>&gt; <BR>&gt;In my installation (sas 9.1 win2k) maximum of lrecl of infile and file is <BR>&gt;1megabyte (2**20) <BR>&gt;lrecl=1048576 is valid <BR>&gt;lrecl=1048577 is invalid <BR>&gt; <BR>&gt;-- <BR>&gt;Richard A. DeVenezia <BR>&gt; <BR>&gt; <BR>&gt; <br></body></html>

Back to: Top of message | Previous page | Main SAS-L page