LISTSERV at the University of Georgia
Menubar Imagemap
Home Browse Manage Request Manuals Register
Previous messageNext messagePrevious in topicNext in topicPrevious by same authorNext by same authorPrevious page (February 2005, week 2)Back to main SAS-L pageJoin or leave SAS-L (or change settings)ReplyPost a new messageSearchProportional fontNon-proportional font
Date:         Mon, 14 Feb 2005 11:22:13 -0500
Reply-To:     David Dorsky <ddorsky@ODH.OHIO.GOV>
Sender:       "SAS(r) Discussion" <SAS-L@LISTSERV.UGA.EDU>
From:         David Dorsky <ddorsky@ODH.OHIO.GOV>
Subject:      Re: PROC SURVEYFREQ

My data set has about 55,000 records, with 90 strata and about 39,000 clusters. I'm subsetting only the variables I need before running PROC SURVEYFREQ.

Table sizes vary, and processing time seems to increase exponentially the more variables and dimensions that I add. For example:

1. A 2-way crosstab with one 2-level column variable and 3 row variable with 12 total levels ran in 25 minutes.

2. However, the processing time more than doubles to nearly an hour if I add a fourth row variable and increase the total number of row levels from 12 to 16;

3. Adding a fifth row variable nearly doubled processing time again to about an hour and a half, even though it added only one additional level. Additional variables increased processing time to more than two hours.

Adding a third dimension, even if only one variable with three levels, appears to more than quadruple processing time:

4. Adding the third dimension to crosstab #1 increased processing time to more than 2 hours.

5. Adding the same third dimension to crosstab #2 failed - it processed for 2.5 hours and then gave me the following error message:

ERROR: Read Access Violation In Task [ SURVEYFREQ ) Exception occurred at (67E2966B) Task Traceback Address Frame (DBGHELP API Version 4.0 rev 5) 67E2966B 04E7FAFC sashost:Main+0x12BEB 53DDC640 04E7FC34 sasqsfrq:mcn_main+0xB640 53DD1DC7 04E7FF88 sasqsfrq:mcn_main+0xDC7 67E227E2 04E7FFA0 sashost:Main+0xBD62 67E26800 04E7FFB4 sashost:Main+0xFD80 77E7D33B 04E7FFEC kernel32:RegisterWaitForInputIdle+0x43

6. A 3-dimensional table with 4 row variables totalling 20 levels ran before the above error, but it took more than 7 hours!

Unfortunately, most of the crosstabs I need to run are 3-way, and some variables have more levels than the above examples. I may be able to reduce total processing time by running one or two variables in a dimension at a time, but that would require many more runs

I did send a data subset and sample program to SAS tech support. They agreed it seemed to run very slow and are assessing it further.

We don't seem have this processing problem with PROC SURVEYMEANS - it runs within seconds most of the time.

Thanks for your reply

Dave Dorsky

On Fri, 11 Feb 2005 15:12:15 -0800, David L. Cassell > >I haven't seen this problem before. (At least, not with SAS 9.1.x - >don't ask about using the experimental version of PROC SURVEYFREQ in >SAS 9.0 !) > >How big are your input data sets? >How big are the tables you are creating? (r by c <by...>) >How many tables are you trying to create at one time? >Have you talked to your SAS site rep or SAS tech support about this? > >David >-- >David Cassell, CSC >Cassell.David@epa.gov >Senior computing specialist >mathematical statistician


Back to: Top of message | Previous page | Main SAS-L page