Date:         Tue, 22 Jun 2004 14:04:45 CEST
Reply-To:     datametric@CLUB-INTERNET.FR
Sender:       "SAS(r) Discussion" <SAS-L@LISTSERV.UGA.EDU>
From:         SUBSCRIBE SAS-L Stephane <datametric@CLUB-INTERNET.FR>
Subject:      Re: read a large flat file
Content-Type: text/plain; charset=ISO-8859-1

Good morning,
I take advantage of the situation to put an another question. I was developping this part of the program when the IT staff proposed to me to send me the data through a pointer instead of sending me a flat file.
The application is based on a UNIX server and I receive datas onto a PC.
I sometimes use the interface between C/dll and SAS so I think it&#8217;s a good idea because the server application hasn&#8217;t to write a flat file somewhere but :
* I noted that the peek function support a length of 32k and, as you can see, I usually receive very big file.
* The place into memory for this kind of pointer will be very important and will decrease performances.

Does the alternative is profitable ?
What is the method to receive a pointer with a size of 32+k ? ? ?

Stéphane



----Message d'origine----
>Date:         Mon, 21 Jun 2004 17:28:00 -0400
>De: "Richard A. DeVenezia" <radevenz@IX.NETCOM.COM>
>Sujet:      Re: read a large flat file
>A: SAS-L@LISTSERV.UGA.EDU
>
>Datametric wrote:
>> Yes it's OK.
>>
>> In reality, this my first step and the second concern the managment
>> of the length of the variables and the special characters into.
>> Even if I use the validvarname = ANY, I can't read my first line with
>> your code because it contains dot separators into the variable names
>> and some columns width is greater than 32 :
>>
>"datesMB.M.A.A10.X.1.Z5.0000.Z01.M.E.B.X*MB.M.A.L50.A.1.Z5.0000.Z06.M.E.B.X*
>>
>MB.M.A.L50.A.1.Z5.0000.Z01.M.E.B.X*MB.M.A.A20.A.1.U4.0000.Z01.M.E.B.X*MB.M.A
>>
>..A20.A.1.U5.1000.Z01.M.E.B.X*MB.M.A.L60.X.1.Z5.0000.Z01.M.E.B.X*MB.M.A.L62.X
>>
>..1.Z5.0000.Z01.M.E.B.X*MB.M.A.A20.A.1.U5.2000.Z01.M.E.B.X*MB.M.A.A20.A.1.U5.
>>
>2100.EUR.M.E.B.X*MB.M.A.A20.A.1.U5.2100.Z01.M.E.B.X*MB.M.A.A20.A.1.U5.2200.E
>>
>UR.M.E.B.X*MB.M.A.A20.A.1.U5.2200.Z01.M.E.B.X*MB.M.A.A20.A.1.U6.1000.Z01.M.E
>> .B.X*MB.M.A.L70.X.1.Z5.0000.Z01.M.E.B.X  and so on" .
>>
>> But you didn't know because I didn't write because I was be able to
>> manage them. So I will use your step in other cases (I have it !!)
>> and load the names into macro-variables or into a table to manage
>> them, in this case. Otherwise, if you have a solution....
>>
>> Anyway, Thanck you for this profitable lesson.
>>
>>
>> Kind regards,
>>
>> Stéphane.
>
>Take Peters excellent example and adapt it.
>If the column names are know to exceed 32 characters, than you will have to
>1. use a array name (serie1...serieN)
>or
>2. map the >32 char name to <=32 char name (compress out everything that is
>not a-z_0-9 as input to a truncatable sequence generator)
>
>Consider 1.  The code has to generate
>
>attriblabel
>....
>  <series-name><X> length=$<some determination of expected length of Xth
>variable in series> label="<Xth delimited value from first line>"
>;
>
>input
>  <series-name>1
>....
>  <series-name><N>
>;
>
>You will need to multipass the infile if you want to measure the data before
>the final data step that inputs the data into a SAS table.
>If you are confident the first <X> (you pick X) rows are a good sample
>containing the maximum length of data values to be encountered throughout
>the entire data file, then you will only have to measure the data based on
>the first X data rows.
>
>
>In my installation (sas 9.1 win2k) maximum of lrecl of infile and file is
>1megabyte (2**20)
>lrecl=1048576 is valid
>lrecl=1048577 is invalid
>
>--
>Richard A. DeVenezia
>http://www.devenezia.com/downloads/sas/samples
>
>