Date: Tue, 22 Jun 2004 14:04:45 CEST
Sender: "SAS(r) Discussion" <SAS-L@LISTSERV.UGA.EDU>
From: SUBSCRIBE SAS-L Stephane <datametric@CLUB-INTERNET.FR>
Subject: Re: read a large flat file
Content-Type: text/plain; charset=ISO-8859-1
I take advantage of the situation to put an another question. I was developping this part of the program when the IT staff proposed to me to send me the data through a pointer instead of sending me a flat file.
The application is based on a UNIX server and I receive datas onto a PC.
I sometimes use the interface between C/dll and SAS so I think it’s a good idea because the server application hasn’t to write a flat file somewhere but :
* I noted that the peek function support a length of 32k and, as you can see, I usually receive very big file.
* The place into memory for this kind of pointer will be very important and will decrease performances.
Does the alternative is profitable ?
What is the method to receive a pointer with a size of 32+k ? ? ?
>Date: Mon, 21 Jun 2004 17:28:00 -0400
>De: "Richard A. DeVenezia" <radevenz@IX.NETCOM.COM>
>Sujet: Re: read a large flat file
>> Yes it's OK.
>> In reality, this my first step and the second concern the managment
>> of the length of the variables and the special characters into.
>> Even if I use the validvarname = ANY, I can't read my first line with
>> your code because it contains dot separators into the variable names
>> and some columns width is greater than 32 :
>> .B.X*MB.M.A.L70.X.1.Z5.0000.Z01.M.E.B.X and so on" .
>> But you didn't know because I didn't write because I was be able to
>> manage them. So I will use your step in other cases (I have it !!)
>> and load the names into macro-variables or into a table to manage
>> them, in this case. Otherwise, if you have a solution....
>> Anyway, Thanck you for this profitable lesson.
>> Kind regards,
>Take Peters excellent example and adapt it.
>If the column names are know to exceed 32 characters, than you will have to
>1. use a array name (serie1...serieN)
>2. map the >32 char name to <=32 char name (compress out everything that is
>not a-z_0-9 as input to a truncatable sequence generator)
>Consider 1. The code has to generate
> <series-name><X> length=$<some determination of expected length of Xth
>variable in series> label="<Xth delimited value from first line>"
>You will need to multipass the infile if you want to measure the data before
>the final data step that inputs the data into a SAS table.
>If you are confident the first <X> (you pick X) rows are a good sample
>containing the maximum length of data values to be encountered throughout
>the entire data file, then you will only have to measure the data based on
>the first X data rows.
>In my installation (sas 9.1 win2k) maximum of lrecl of infile and file is
>lrecl=1048576 is valid
>lrecl=1048577 is invalid
>Richard A. DeVenezia