Hi Jonathon
Here is a summary of the results, with the
value of WORKSPAVE set at 6148, 100000, 500000, 1000000.
|
WORKSPACE |
Elapsed time,
sort |
CPU time, sort |
Whole task, CPU
time |
Whole task,
elapsed time |
No. sorts |
Merge, elapsed
time |
Merge, CPU time |
|
6148 |
24.60 |
35.80 |
55.28 |
48.66 |
8.00 |
24.00 |
19.50 |
|
100000 |
24.10 |
37.20 |
56.88 |
50.13 |
8.00 |
25.90 |
19.60 |
|
500000 |
34.90 |
60.90 |
71.22 |
50.76 |
2.00 |
15.70 |
10.30 |
|
1000000 |
28.70 |
46.30 |
56.41 |
43.56 |
2.00 |
14.80 |
10.00 |
Elapsed overall time seems to get worse
with increasing memory (counterintuitive) whilst merge times go down as would
be expected from having to merge fewer subsets. It tends to support the advice in
the manual that you should just let SPSS work out the best way of doing it! ;-)
It is very puzzling (well, to me anyway)
that it should make things worse to have more memory available.
Below are the detailed reports from each
run
Here’s the report from the listing
with WORKSPACE set to 1000000:
sort cases locality_name, street_name,st_type,street_suffix
number_first,number_last,flat_number,LOT_NUMBER.
Row size: 856 bytes
Row count: 932355
Uncompressed file size: 761.1MB
Elapsed time for sort phase: 28.7 seconds
Processor time for sort phase: 46.3 seconds
Total size of files written by sort phase: 252.3MB
Sort #1
Memory budget: 500000KB
Row limit: 1340314
Rows sorted: 339763
Input time: 2.8 seconds
Sort time : 11.9 seconds
Output time: 3.7 seconds
Sort #2
Memory budget: 500000KB
Row limit: 592592
Rows sorted: 592592
Input time: 4.7 seconds
Sort time : 19.0 seconds
Output time: 4.9 seconds
Merge # 1
Sequences merged: 2
Elapsed time: 14.8 seconds
Processor time: 10.0 seconds
Preceding task required 56.41 seconds CPU time; 43.56
seconds elapsed.
Below is the same section after re-setting WORKSPACE to default
value of 6148
Row size: 856 bytes
Row count: 932355
Uncompressed file size: 761.1MB
Elapsed time for sort phase: 24.6 seconds
Processor time for sort phase: 35.8 seconds
Total size of files written by sort phase: 252.3MB
Sort #1
Memory budget: 51200KB
Row limit: 60681
Rows sorted: 60681
Input time: 0.7 seconds
Sort time : 1.6 seconds
Output time: 1.1 seconds
Sort #2
Memory budget: 51200KB
Row limit: 137248
Rows sorted: 129980
Input time: 1.0 seconds
Sort time : 3.6 seconds
Output time: 2.0 seconds
Sort #3
Memory budget: 51200KB
Row limit: 134089
Rows sorted: 134089
Input time: 1.1 seconds
Sort time : 3.6 seconds
Output time: 1.3 seconds
Sort #4
Memory budget: 51200KB
Row limit: 136533
Rows sorted: 136533
Input time: 1.1 seconds
Sort time : 3.8 seconds
Output time: 2.2 seconds
Sort #5
Memory budget: 51200KB
Row limit: 137248
Rows sorted: 137248
Input time: 1.1 seconds
Sort time : 3.8 seconds
Output time: 2.1 seconds
Sort #6
Memory budget: 51200KB
Row limit: 137248
Rows sorted: 137248
Input time: 1.0 seconds
Sort time : 4.0 seconds
Output time: 1.3 seconds
Sort #7
Memory budget: 51200KB
Row limit: 137608
Rows sorted: 130862
Input time: 1.0 seconds
Sort time : 3.8 seconds
Output time: 3.0 seconds
Sort #8
Memory budget: 51200KB
Row limit: 136889
Rows sorted: 65714
Input time: 0.5 seconds
Sort time : 1.9 seconds
Output time: 1.3 seconds
Merge # 1
Sequences merged: 8
Elapsed time: 24.0 seconds
Processor time: 19.5 seconds
Preceding task required 55.28 seconds CPU time; 48.66
seconds elapsed.
Here it is set to 500,000
Row size: 856 bytes
Row count: 932355
Uncompressed file size: 761.1MB
Elapsed time for sort phase: 34.9 seconds
Processor time for sort phase: 60.9 seconds
Total size of files written by sort phase: 252.3MB
Sort #1
Memory budget: 250000KB
Row limit: 296296
Rows sorted: 296296
Input time: 2.6 seconds
Sort time : 9.0 seconds
Output time: 2.5 seconds
Sort #2
Memory budget: 250000KB
Row limit: 664935
Rows sorted: 636059
Input time: 5.0 seconds
Sort time : 21.8 seconds
Output time: 5.5 seconds
Merge # 1
Sequences merged: 2
Elapsed time: 15.7 seconds
Processor time: 10.3 seconds
Preceding task required 71.22 seconds CPU time; 50.76
seconds elapsed.
And here it is set to 100000
Row size: 856 bytes
Row count: 932355
Uncompressed file size: 761.1MB
Elapsed time for sort phase: 24.1 seconds
Processor time for sort phase: 37.2 seconds
Total size of files written by sort phase: 252.3MB
Sort #1
Memory budget: 51200KB
Row limit: 60681
Rows sorted: 60681
Input time: 0.7 seconds
Sort time : 1.8 seconds
Output time: 1.9 seconds
Sort #2
Memory budget: 51200KB
Row limit: 137248
Rows sorted: 129980
Input time: 1.1 seconds
Sort time : 3.5 seconds
Output time: 2.7 seconds
Sort #3
Memory budget: 51200KB
Row limit: 134089
Rows sorted: 134089
Input time: 1.1 seconds
Sort time : 3.7 seconds
Output time: 1.2 seconds
Sort #4
Memory budget: 51200KB
Row limit: 136533
Rows sorted: 136533
Input time: 1.1 seconds
Sort time : 4.2 seconds
Output time: 1.2 seconds
Sort #5
Memory budget: 51200KB
Row limit: 137248
Rows sorted: 137248
Input time: 1.1 seconds
Sort time : 4.0 seconds
Output time: 1.4 seconds
Sort #6
Memory budget: 51200KB
Row limit: 137248
Rows sorted: 137248
Input time: 1.1 seconds
Sort time : 4.2 seconds
Output time: 1.1 seconds
Sort #7
Memory budget: 51200KB
Row limit: 137608
Rows sorted: 130862
Input time: 1.1 seconds
Sort time : 4.1 seconds
Output time: 1.3 seconds
Sort #8
Memory budget: 51200KB
Row limit: 136889
Rows sorted: 65714
Input time: 0.6 seconds
Sort time : 2.0 seconds
Output time: 0.8 seconds
Merge # 1
Sequences merged: 8
Elapsed time: 25.9 seconds
Processor time: 19.6 seconds
Preceding task required 56.88 seconds CPU time; 50.13
seconds elapsed.
Adrian
Barnett
Project Officer
Educational
Measurement and Analysis
Data and
Educational Measurement
DECS
ph 82261080
From:
Jon Fry [mailto:JonathanFry@us.ibm.com]
Sent: Thursday, 13 May 2010 1:18
AM
To: Barnett, Adrian (DECS)
Cc: '
Subject: RE: CPU Specifications
used for SPSS
For the two-sort version, what are the reported read, sort, and write times?
Jonathan