Date: Tue, 22 Jun 2004 10:35:52 CEST
Reply-To: datametric@CLUB-INTERNET.FR
Sender: "SAS(r) Discussion" <SAS-L@LISTSERV.UGA.EDU>
From: SUBSCRIBE SAS-L Stephane <datametric@CLUB-INTERNET.FR>
Subject: Re: read a large flat file
Content-Type: text/html; charset=ISO-8859-1
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<HTML>
<HEAD><TITLE></TITLE>
<STYLE>
body, table, tr, td, p {font-family: Verdana;font-size:12px;margin: 0px 0px 0px 0px}
.bgtabl {BACKGROUND-REPEAT: no-repeat}
</STYLE>
</HEAD>
<BODY bgProperties="fixed" bgcolor="#FFFFFF" background="http://flashimg.club-internet.fr/flashmail/vide.gif">
Good morning, <BR> <BR>I take advantage of the situation to put an another question. <BR> <BR>I was developping this part of the program when the IT staff proposed to me to send me the data through a pointer instead of sending me a flat file. The application is based on a UNIX server and I receive datas onto a PC. <BR> <BR>I sometimes use the interface between C/dll and SAS so I think it’s a good idea because the server application hasn’t to write a flat file somewhere but : <BR>*
I noted that the peek function support a length of 32k and, as you can see, I usually receive very big file. <BR>* the place into memory for this kind of pointer will be very important and will decrease performances. <BR> <BR>Does the method is profitable ? <BR> <BR>What is the method to receive a pointer with a size of 32+k ? ? ? <BR> <BR>Stéphane. <BR><BR><BR><BR>----Message d'origine---- <BR>>Date: Mon, 21 Jun 2004 17:28:00 -0400 <BR>>De: "Richard A. DeVenezia" <RADEVENZ@IX.NETCOM.
COM><BR>>Sujet: Re: read a large flat file <BR>>A: SAS-L@LISTSERV.UGA.EDU <BR>> <BR>>Datametric wrote: <BR>>> Yes it's OK. <BR>>> <BR>>> In reality, this my first step and the second concern the managment <BR>>> of the length of the variables and the special characters into. <BR>>> Even if I use the validvarname = ANY, I can't read my first line with <BR>>> your code because it contains dot separators into the variable names <BR>>> and some columns width
is greater than 32 : <BR>>> <BR>>"datesMB.M.A.A10.X.1.Z5.0000.Z01.M.E.B.X*MB.M.A.L50.A.1.Z5.0000.Z06.M.E.B.X* <BR>>> <BR>>MB.M.A.L50.A.1.Z5.0000.Z01.M.E.B.X*MB.M.A.A20.A.1.U4.0000.Z01.M.E.B.X*MB.M.A <BR>>> <BR>>..A20.A.1.U5.1000.Z01.M.E.B.X*MB.M.A.L60.X.1.Z5.0000.Z01.M.E.B.X*MB.M.A.L62.X <BR>>> <BR>>..1.Z5.0000.Z01.M.E.B.X*MB.M.A.A20.A.1.U5.2000.Z01.M.E.B.X*MB.M.A.A20.A.1.U5. <BR>>> <BR>>2100.EUR.M.E.B.X*MB.M.A.A20.A.1.U5.2100.Z01.M.E.B.X*MB.M.A.A20.A.1.U5.2200.E
<BR>>> <BR>>UR.M.E.B.X*MB.M.A.A20.A.1.U5.2200.Z01.M.E.B.X*MB.M.A.A20.A.1.U6.1000.Z01.M.E <BR>>> .B.X*MB.M.A.L70.X.1.Z5.0000.Z01.M.E.B.X and so on" . <BR>>> <BR>>> But you didn't know because I didn't write because I was be able to <BR>>> manage them. So I will use your step in other cases (I have it !!) <BR>>> and load the names into macro-variables or into a table to manage <BR>>> them, in this case. Otherwise, if you have a solution.... <BR>>> <BR>>
> Anyway, Thanck you for this profitable lesson. <BR>>> <BR>>> <BR>>> Kind regards, <BR>>> <BR>>> Stéphane. <BR>> <BR>>Take Peters excellent example and adapt it. <BR>>If the column names are know to exceed 32 characters, than you will have to <BR>>1. use a array name (serie1...serieN) <BR>>or <BR>>2. map the >32 char name to <=32 char name (compress out everything that is <BR>>not a-z_0-9 as input to a truncatable sequence generator) <BR>>
<BR>>Consider 1. The code has to generate <BR>> <BR>>attriblabel <BR>>.... <BR>> <SERIES-NAME><X>length=$<SOME <br Xth of length expected determination>>variable in series> label="<XTH line first from value delimited>" <BR>>; <BR>> <BR>>input <BR>> <SERIES-NAME>1 <BR>>.... <BR>> <SERIES-NAME><N><BR>>; <BR>> <BR>>You will need to multipass the infile if you want to measure the data before <BR>>the final data step that inputs the data into a SAS table.
<BR>>If you are confident the first <X>(you pick X) rows are a good sample <BR>>containing the maximum length of data values to be encountered throughout <BR>>the entire data file, then you will only have to measure the data based on <BR>>the first X data rows. <BR>> <BR>> <BR>>In my installation (sas 9.1 win2k) maximum of lrecl of infile and file is <BR>>1megabyte (2**20) <BR>>lrecl=1048576 is valid <BR>>lrecl=1048577 is invalid <BR>> <BR>>-- <BR>>Richard A. DeVenezia
<BR>>http://www.devenezia.com/downloads/sas/samples <BR>> <BR>> <br></body></html>
| 
