Date:         Fri, 8 May 2009 15:09:21 -0700
Reply-To:     "Choate, Paul@DDS" <pchoate@DDS.CA.GOV>
Sender:       "SAS(r) Discussion" <SAS-L@LISTSERV.UGA.EDU>
From:         "Choate, Paul@DDS" <pchoate@DDS.CA.GOV>
Subject:      Re: csv or txt file works in dev but fails in SIT
Comments: To: Dianne@freddie.mac
In-Reply-To:  A<200905082118.n48KD0B4001082@malibu.cc.uga.edu>
Content-Type: text/plain; charset="us-ascii"

Glad to be of help Dianna -

I usually use SAS to view hex. HexMad is a great freeware Windows app
for just viewing Hex in a file. http://www.google.com/search?q=HexMad

Something like this will tell you the row, variable, and position in the
variable of your non-printing characters...

data yourfile;
var1='some stuff'||'000203'x||'more stuff';
var2='some stuff'||'0904'x||'more stuff';
var3='some stuff'||'850709'x||'more stuff';
output;
var1='some stuff'||'900203'x||'more stuff';
var2='some stuff'||'0904'x||'more stuff';
var3='some stuff'||'05078A'x||'more stuff';
output;
run;


data baddata(keep=line var position byte hexbyte);
     set yourfile;
     array charvars $200 _character_;
     line=_n_;
     length var $32 byte $1;
     do over charvars;
          if ~missing(compress(charvars,,'sw'))
          then do;
               var=vname(charvars);
               do position = 1 to length(charvars);
               byte=substr(charvars,position,1);
               if (byte<'20'x or byte>'7F'x) and byte ne '09'x then do;
                    hexbyte=put(byte,$hex2.);
                    output;
                    end;
               end;
          end;
     end;
run;

This probably isn't your problem - but take a look at
http://www.sascommunity.org/wiki/IGNOREDOSEOF_Option

This has tripped me up before...


Paul Choate
DDS Data Extraction
(916) 654-2160

-----Original Message-----
From: SAS(r) Discussion [mailto:SAS-L@LISTSERV.UGA.EDU] On Behalf Of
Dianne Rhodes
Sent: Friday, May 08, 2009 2:19 PM
To: SAS-L@LISTSERV.UGA.EDU
Subject: Re: csv or txt file works in dev but fails in SIT

On Fri, 8 May 2009 13:46:45 -0700, Choate, Paul@DDS <pchoate@DDS.CA.GOV>
wrote:

>Hi Dianna -
>
>Why not array your character variables and strip out the printing and
>space characters, and look and see what's left?
>

Thanks, we'll try some thing like this.  It's clearly an unprintable /
seeable hex string that's gotten in there.  The other programmer who is
working on this found it by manually editing the file.  Is there a text
editor for Windows that would let you see hex?  I know you can do this
with ISPF but I don't think we have that.

Dianne