Date: Tue, 30 May 2006 17:00:29 -0500
Reply-To: "Peck, Jon" <peck@spss.com>
Sender: "SPSSX(r) Discussion" <SPSSX-L@LISTSERV.UGA.EDU>
From: "Peck, Jon" <peck@spss.com>
Subject: Re: Running a single syntax file using MANY data files
Content-Type: text/plain; charset="iso-8859-1"
Skimming through a lot of email on this topic, I did not see what I think is actually the easiest and most robust way to tackle problems like this. (If you are a regular reader, you can guess what I am going to say.)
Use BEGIN PROGRAM and Python. One of the examples in my Directions PowerPoint, which is available on the new Developer Central site, which replaces the SPSS Code Center, (www.spss.com/devcentral) illustrates driving syntax from all the sav files found in a particular location. I think it is example 9 in that document, which merges all the sav files in a particular directory.
With this approach, you would, in the syntax stream, discover the files matching some filespec (with wildcards if desired), and iterate your SPSS syntax over these files.
There is a viewer module that allows you to control the Viewer, including the standard SPSS exports. I don't know how you are generating the pdfs (pdf export will be a standard feature in SPSS 15), but that can be handled with automation calls from the Python code also.
I can provide more details on how to go about all this if this is the way you want to go.
Regards,
Jon Peck
SPSS
-----Original Message-----
From: SPSSX(r) Discussion on behalf of Nick White
Sent: Mon 5/29/2006 8:54 PM
To: SPSSX-L@LISTSERV.UGA.EDU
Subject: Re: [SPSSX-L] Running a single syntax file using MANY data files
Adrian, thank you very much for your ideas! Final output report has been
thoroughly debugged. Input data was very nicely cleaned before we received
it (it is survey data that was collected via a webpage... the programmer did
a very nice job and we have found no invalid data... so far...). The .sav
files are all in the same directory, but are named after schools (definitely
not a nice workable progression). That's okay, though, because I am printing
the output files to PDF's with the same names as the schools (which is what
we will deliver to the state department of education). I've run and proofed
reports for 10 different schools, so I am optimistic (though you will never
catch me saying "NOTHING can go wrong!!!") While I wish I had a nice text
editor, I'll likely be using Word. I now have 3 approaches to try (not
counting my script which didn't even get off the ground...).
Thanks!
Nick White
-----Original Message-----
From: Barnett, Adrian (HEALTH) [mailto:adrian.barnett@health.sa.gov.au]
Sent: Monday, May 29, 2006 8:58 PM
To: Nick White; SPSSX-L@LISTSERV.UGA.EDU
Subject: RE: Running a single syntax file using MANY data files
Hi Nick
If your files are all in the same directory and have convenient names
like FILE001 to FILE800, you can write a macro which will do the job. A
naming convention like the above will make it much easier to express he
file list succinctly.
If your files are all over the place in different directories, it will
be much harder and I'm not sure what I'd suggest (Maybe moving them into
the same directory?). If they are all in the same directory and they
don't have names forming a logical progression like the above, or if you
don't fancy the dubious pleasures of macro-writing, you could do it in
the following way.
Open a command window and type:
Dir *.sav > files.sps
That will write a list of all the files with a .sav extension into a
file called files.sps. This will work best if you only have the files
concerned in that directory. Otherwise you will have to edit out the
unwanted file names.
Here is the beginning of the file you get by running the above command
in the SPSS directory:
Volume in drive C is Local Drive
Volume Serial Number is 5CE7-523B
Directory of C:\Program Files\SPSS
16/09/2002 10:13 103,627 1991 U.S. General Social Survey.sav
30/11/1999 05:50 664 AML survival.sav
25/10/1999 10:03 6,664 anorectic.sav
30/11/1999 05:49 768 Anxiety 2.sav
Using a text editor or word processor, delete all the stuff in front of
the file name and the preamble at the top about volume and directory
etc. (MS Word can, I think, do columnwise selections and deletions. A
decent text editor will too and is much better suited to tasks like
this.)
Prefix the file names with
INSERT FILE='
Add a closing ' to the end of each filename and any other options that
seem relevant to you (refer to the syntax manual for the INSERT command
for these).
Save your file with a .sps extension and open it in the SPSS syntax
editor and run it. You can also use the SPSS Production facility on the
same file with a few alterations to it. You can experiment for yourself
creating a production job, saving the output and then opening it in an
editor to see what these other changes are. I would not do this till it
was clear the whole job had been properly debugged and was something you
were going to run regularly though.
Whichever option you use, you are still going to have an awful lot of
work to do once all this has run, as you will have 800 sets of results.
Debugging will be difficult in the event there are peculiarities in the
data of only SOME files so you will have to check the outputs carefully.
You might want to put in tests for certain unwanted conditions, so you
can search for the results in the output file. Examples might be values
out of range, two or more variables not bearing the correct relationship
(e.g. if variable A is equal to a certain value then variables B and C
can't be less than such-and-such), no cases in some expected value
range and so on. You'd definitely want to use the FIND command in the
output window to search for the strings ERROR and WARNING to make sure
you find any problems you might miss if you search by eye. I'd start off
running just one or a few files first for the initial debugging and add
the rest once you are confident it's all working as you want.
Good luck.
Regards
Adrian
--
Adrian Barnett
Research & Information Officer Ph: +61 8 82266615
Research, Analysis and Evaluation Fax: +61 8
82267088
Strategic Planning and Research Branch
Strategic Planning and Population Health Division
Department of Health
God is good but don't dance in a currach. - Irish proverb
This e-mail may contain confidential information, which also may be
legally privileged. Only the intended recipient(s) may access, use,
distribute or copy this e-mail. If this e-mail is received in error,
please inform the sender by return e-mail and delete the original. If
there are doubts about the validity of this message, please contact the
sender by telephone. It is the recipient's responsibility to check the
e-mail and any attached files for viruses.
-----Original Message-----
From: SPSSX(r) Discussion [mailto:SPSSX-L@LISTSERV.UGA.EDU] On Behalf Of
Nick White
Sent: Monday, 29 May 2006 4:14
To: SPSSX-L@LISTSERV.UGA.EDU
Subject: Running a single syntax file using MANY data files
I have a syntax file and associated scripts that run an analysis and
produce a report in pdf format that has the same name as the data file.
I wish to run this program using about 800 data files (all in .sav
format). What is the easiest way to run the syntax file against all 800
files? Production facility doesn't seem to be the answer. I am not
interested in sitting at the computer for 40+ hours entering data file
names.
Thanks,
Nick White
