Date: Thu, 6 Apr 2000 19:17:38 +1200
Reply-To: Don Stanley <don_stanley@XTRA.CO.NZ>
Sender: "SAS(r) Discussion" <SAS-L@LISTSERV.UGA.EDU>
From: Don Stanley <don_stanley@XTRA.CO.NZ>
Organization: Xtra
Subject: Re: MVS -- U1319, No MKLES
Content-Type: text/plain; charset=us-ascii
This should be tracked to SAS. Its of interest to any shop considering going to V8
and already having high memory utilisation in V6.12.
I've got this nightmare now of our guys going to V8 and finding jobs working then
not working then working etc etc etc because of less memory being available. Tell
me I'm just paranoid ... SAS wouldn't steal our memory now would they ...
Don
Puddin' Man wrote:
> I've been told that many software products will:
>
> 1.) Use more memory than "necessary" if it is
> available.
> 2.) Often fail to release memory when it is
> no longer needed.
>
> To my knowledge SAS is guilty of both. So I was
> glad to hear of OPTIONS MINSTG; in Paul's post
> (thanks Paul).
>
> 'Course I had to test it. For OS/390 V6 I got:
>
> NOTE: Running on IBM Model 9672 Serial Number xxxxxx,
> *------------------------------------------------------------------*
> *- SAS Version 6.09 Enhanced Put Level TS450 -*
> *------------------------------------------------------------------*
> NOTE: The initialization phase used 0.06 CPU seconds and 2759K.
> 1 options nominstg;
> 2 data _null_; array a(300000) _temporary_; run;
> NOTE: The DATA statement used 0.02 CPU seconds and 5628K.
> 3 data _null_; array a(200000) _temporary_; run;
> NOTE: The DATA statement used 0.01 CPU seconds and 5628K.
> 4 data _null_; array a(100000) _temporary_; run;
> NOTE: The DATA statement used 0.01 CPU seconds and 5628K.
> 5 data _null_; x=1 ; run;
> NOTE: The DATA statement used 0.00 CPU seconds and 5628K.
>
> 6 options minstg;
> 7 data _null_; array a(300000) _temporary_; run;
> NOTE: The DATA statement used 0.01 CPU seconds and 5664K.
> 8 data _null_; array a(200000) _temporary_; run;
> NOTE: The DATA statement used 0.01 CPU seconds and 4824K.
> 9 data _null_; array a(100000) _temporary_; run;
> NOTE: The DATA statement used 0.01 CPU seconds and 4044K.
> 10 data _null_; x=1 ; run;
> NOTE: The DATA statement used 0.01 CPU seconds and 3260K.
>
> which is comparable to Paul's result.
>
> But when I tested with V8, I got:
>
> *------------------------------------------------------------------*
> *- SAS Version 8.00 Enhanced TS M0 -*
> *------------------------------------------------------------------*
> 1 options nominstg;
> 2 data _null_; array a(300000) _temporary_; run;
> NOTE: The DATA statement used 0.02 CPU seconds and 9247K.
> 3 data _null_; array a(200000) _temporary_; run;
> NOTE: The DATA statement used 0.01 CPU seconds and 9247K.
> 4 data _null_; array a(100000) _temporary_; run;
> NOTE: The DATA statement used 0.01 CPU seconds and 9247K.
> 5 data _null_; x=1 ; run;
> NOTE: The DATA statement used 0.00 CPU seconds and 9247K.
>
> 6 options minstg;
> 7 data _null_; array a(300000) _temporary_; run;
> NOTE: The DATA statement used 0.01 CPU seconds and 9247K.
> 8 data _null_; array a(200000) _temporary_; run;
> NOTE: The DATA statement used 0.01 CPU seconds and 9163K.
> 9 data _null_; array a(100000) _temporary_; run;
> NOTE: The DATA statement used 0.01 CPU seconds and 9163K.
> 10 data _null_; x=1 ; run;
> NOTE: The DATA statement used 0.01 CPU seconds and 9163K.
>
> These are drastically different numbers. The advantage for
> MINSTG almost disappears. And V8 is using over 60% more
> internal storage.
>
> Paul, did you use V6 for your test? Can you test with
> V6 _and_ V8?
>
> Zalut,
> Puddin'
>
> On Wed, 05 April 2000, Paul Dorfman wrote:
>
> >
> > Don,
> >
> > It may be as simple as memory shortage, or a lot peskier problem having to
> > do with a SAS bug.
> >
> > >From: Don Stanley <don_stanley@XTRA.CO.NZ>
> > >
> > >I've struck this error many times over the years.
> >
> > Don,
> >
> > Ditto. It may be as simple as memory shortage, or a lot peskier problem
> > having to do with a SAS bug which SI does admit to.
> >
> > Sometimes I've found
> > >my code needs tweaking, like when I had a massive format in a CNTLIN
> > >dataset that wasn't sorted by the START variable. Sorting got round the
> > >problem. SAS themselves document many usage notes and zaps relating to
> > >this. As I understand it, its generally an indicator of a memory
> > >shortage. But I've found no-one, not even at SAS, who can tell me what
> > >MKLE's means.
> >
> > That sorting can actually decrease a format's memory usage is puzzling. Not
> > so long ago, I was struggling with the same situation. First I took the lazy
> > Susan approach and called SI. They sent me a zap for this bug. However, it
> > was in December 1999, and the system folks had no desire to install
> > _anything_ before after Y2K, so I ended up taking the industriuos Paul path,
> > rewrote the entire thingie, and ran it in one-third the memory.
> >
> > >Can someone enlighten me?
> >
> > Not me. I asked the same question the SI MVS developer I was talking to
> > regarding the aforementioned bug but never got to the point of deciphering
> > the abbreviation. In my internal dictionary, it is mapped to the slot 'bad
> > memory thing', somewhere next to S0C4.
> >
> > >Just as a point of interest, when I strike this problem, I almost always
> > >use the MEMSIZE=0 invocation option as a first try at circumventing it.
> > >I call this a 95% fix, the other 5% of the time I end up rewriting the
> > >code. Today we put a new job into production. We have run it in test 20,
> > >maybe 30 times. Bam. U1319 immediately. Only differences I can tell are
> > >it runs under CONTROL-M (a scheduler) and it runs in a production
> > >service class. The step that failed appears to be setting 13 datasets
> > >(SET A B C ....), all with same structure, and all having a fairly small
> > >number of variables. About 4000 records per dataset. I switched on
> > >MEMSIZE=0 at invocation and its fine. I'd love to know what causes this
> > >in the first place.
> >
> > In this particular case, you probably ran out of memory for buffers with 8M
> > of real storage specified as default. MEMSIZE=0 of course fixed it by
> > grabbing as much memory as the Big Iron Brother is willing to give you.
> > However, MVS developers in SI insist adamantly that MEMSIZE=0 must never be
> > used, and that REGION= and MEMSIZE= must be set to the same value you have
> > to estimate beforehand.
> >
> > Another option I have found to be absolutely indispensable when memory is at
> > stake is MINSTG ('minimize storage'; specified either at invocation or in
> > the program). It causes a SAS step suspend all the real storage it has
> > allocated upon the completion of the step. Otherwise, if the subsequent step
> > requires more memory than the current one, memory usage keeps piling up.
> > Watch this. OPTION MINSTG:
> >
> > 30 data _null_; array a(200000) _temporary_; run;
> > NOTE: The DATA statement used 0.02 CPU seconds and 4561K.
> >
> > 31 data _null_; array a(100000) _temporary_; run;
> > NOTE: The DATA statement used 0.01 CPU seconds and 3781K.
> >
> > 32 data _null_; array a(300000) _temporary_; run;
> > NOTE: The DATA statement used 0.02 CPU seconds and 5341K.
> >
> > 33 data _null_; array a(200000) _temporary_; run;
> > NOTE: The DATA statement used 0.02 CPU seconds and 4561K.
> >
> > As you see, each step allocates exactly as much memory as it needs. Now same
> > thing with OPTION NOMINSTG:
> >
> > 42 data _null_; array a(200000) _temporary_; run;
> > NOTE: The DATA statement used 0.01 CPU seconds and 4561K.
> >
> > 43 data _null_; array a(100000) _temporary_; run;
> > NOTE: The DATA statement used 0.01 CPU seconds and 4561K.
> >
> > 44 data _null_; array a(300000) _temporary_; run;
> > NOTE: The DATA statement used 0.02 CPU seconds and 7027K.
> >
> > 45 data _null_; array a(200000) _temporary_; run;
> > NOTE: The DATA statement used 0.01 CPU seconds and 7027K.
> >
> > With 300000 elements in A, the step needs only 5341K, yet uses 7027 since
> > the previous step's memory has not been released. MINSTG is actually
> > documented in one of the v6 tech reports. They say that NOMINSTG provides
> > some performance advantages by keeping the load modules in memory all the
> > time, but I have not found any advantage to it, with the exception of having
> > fun to rerun jobs bombed with the 'bad memory thing'.
> >
> > Kind regards,
> > =====================
> > Paul M. Dorfman
> > Jacksonville, Fl
> > =====================
> >
> >
> > >Don
> > >--
> > >Don Stanley, B.SC, Dip O.R.S, MNZCS Director, Sysware Consulting
> > >Group
> > > Box 634, Wellington, NEW ZEALAND
> > >
> > > http://www.sysware.co.nz
> > >EMAIL:: don_stanley@xtra.co.nz
> > >http://www.geocities.com/don_stanley_nz/don_home.htm
> > >Genealogy:: http://www.geocities.com/don_stanley_nz/family.htm
> >
> > ______________________________________________________
> > Get Your Private, Free Email at http://www.hotmail.com
>
> *****************************************************
> *** Puddin' Man *** pudding_man@altavista.com ***
> *****************************************************;
>
> ____________________________________________________________________
>
> For the largest MP3 index on the Web, go to http://mp3.altavista.com
>
> ____________________________________________________________________
--
Don Stanley, B.SC, Dip O.R.S, MNZCS Director, Sysware Consulting Group
Box 634, Wellington, NEW ZEALAND
http://www.sysware.co.nz
EMAIL:: don_stanley@xtra.co.nz
http://www.geocities.com/don_stanley_nz/don_home.htm
Genealogy:: http://www.geocities.com/don_stanley_nz/family.htm
| 
