Date: Mon, 14 Feb 2005 11:22:13 -0500
Reply-To: David Dorsky <ddorsky@ODH.OHIO.GOV>
Sender: "SAS(r) Discussion" <SAS-L@LISTSERV.UGA.EDU>
From: David Dorsky <ddorsky@ODH.OHIO.GOV>
Subject: Re: PROC SURVEYFREQ
My data set has about 55,000 records, with 90 strata and about 39,000
clusters. I'm subsetting only the variables I need before running PROC
SURVEYFREQ.
Table sizes vary, and processing time seems to increase exponentially the
more variables and dimensions that I add. For example:
1. A 2-way crosstab with one 2-level column variable and 3 row variable
with 12 total levels ran in 25 minutes.
2. However, the processing time more than doubles to nearly an hour if I
add a fourth row variable and increase the total number of row levels from
12 to 16;
3. Adding a fifth row variable nearly doubled processing time again to
about an hour and a half, even though it added only one additional level.
Additional variables increased processing time to more than two hours.
Adding a third dimension, even if only one variable with three levels,
appears to more than quadruple processing time:
4. Adding the third dimension to crosstab #1 increased processing time to
more than 2 hours.
5. Adding the same third dimension to crosstab #2 failed - it processed for
2.5 hours and then gave me the following error message:
ERROR: Read Access Violation In Task [ SURVEYFREQ )
Exception occurred at (67E2966B)
Task Traceback
Address Frame (DBGHELP API Version 4.0 rev 5)
67E2966B 04E7FAFC sashost:Main+0x12BEB
53DDC640 04E7FC34 sasqsfrq:mcn_main+0xB640
53DD1DC7 04E7FF88 sasqsfrq:mcn_main+0xDC7
67E227E2 04E7FFA0 sashost:Main+0xBD62
67E26800 04E7FFB4 sashost:Main+0xFD80
77E7D33B 04E7FFEC kernel32:RegisterWaitForInputIdle+0x43
6. A 3-dimensional table with 4 row variables totalling 20 levels ran
before the above error, but it took more than 7 hours!
Unfortunately, most of the crosstabs I need to run are 3-way, and some
variables have more levels than the above examples. I may be able to
reduce total processing time by running one or two variables in a dimension
at a time, but that would require many more runs
I did send a data subset and sample program to SAS tech support. They
agreed it seemed to run very slow and are assessing it further.
We don't seem have this processing problem with PROC SURVEYMEANS - it runs
within seconds most of the time.
Thanks for your reply
Dave Dorsky
On Fri, 11 Feb 2005 15:12:15 -0800, David L. Cassell
>
>I haven't seen this problem before. (At least, not with SAS 9.1.x -
>don't ask about using the experimental version of PROC SURVEYFREQ in
>SAS 9.0 !)
>
>How big are your input data sets?
>How big are the tables you are creating? (r by c <by...>)
>How many tables are you trying to create at one time?
>Have you talked to your SAS site rep or SAS tech support about this?
>
>David
>--
>David Cassell, CSC
>Cassell.David@epa.gov
>Senior computing specialist
>mathematical statistician
