Date: Thu, 11 Feb 2010 17:17:37 +0100
Reply-To: Derek Willemsen <DerekWillemsen@invicta.nl>
Sender: "SPSSX(r) Discussion" <SPSSX-L@LISTSERV.UGA.EDU>
From: Derek Willemsen <DerekWillemsen@invicta.nl>
Subject: Re: Speeding up aggregate
Thanks for your reply. I've tested it and it's still getting slow after
9-10 million records.. The first 9 million were processed in a few
seconds so I was hopeful, but after a while it slowed down and it was
processing a couple of hundred records a second (instead of
Van: SPSSX(r) Discussion [mailto:SPSSX-L@LISTSERV.UGA.EDU] Namens Mike
Verzonden: donderdag 11 februari 2010 16:53
Onderwerp: Re: Speeding up aggregate
Does it still take that long if you use a file that only has the four
variables and perhaps a case ID?
New York University
----- Original Message -----
From: Derek Willemsen <mailto:DerekWillemsen@invicta.nl>
Sent: Thursday, February 11, 2010 10:31 AM
Subject: Speeding up aggregate
I have a dataset which contains 16 million records. I need to
count how many records there are on 4 break variables so I use an simple
aggregate with ADDVARIABLES mode.
/BREAK=VAR1 VAR2 VAR3 VAR4
The first couple of million goes fast, but after 11 million
records the aggregation is getting really slow. It takes ages to finish
the last 5 million records. Normally it takes about 2,5 hours to finish
Is there a way speed this process up?
(I have 2GB internal memory)
Thanks in advance!