Date: Wed, 9 Jan 2008 10:38:48 0600
ReplyTo: "data _null_," <datanull@GMAIL.COM>
Sender: "SAS(r) Discussion" <SASL@LISTSERV.UGA.EDU>
From: "data _null_," <datanull@GMAIL.COM>
Subject: Re: Can you do array processing and a proc nlp from within proc
iml?
InReplyTo: <200801091542.m09Bl6dU027597@malibu.cc.uga.edu>
ContentType: text/plain; charset=UTF8
1) How many rows are in the maxtix that is output from RANDMULTINOMIAL?
2) What are the columns names in the data set created, thay need to be
made to match the names you are using in PROC NLP.
3) How many rows from RANDMULTINOMIAL are needed for each "fit" using PROC NLP.
4) What is _OVERALL for?
On Jan 9, 2008 9:42 AM, Randall Powers <powers_r@bls.gov> wrote:
> Thanks again, that sounds promising. The log is giving me the follow error
> (below, each iteration has the same error messages). My program is also
> below. What do I need to alter to correct the errors? Previously, I had
> manual inputs of n0, n11....n24, and the program worked. So it's obviously
> just a question of somehow letting SAS know that the values in the dataset
> are the values of n0, n11....n24.
>
>
> NOTE: Your code contains 5 program statements.
> NOTE: Gradient is computed using analytic formulas.
> NOTE: Hessian is computed using analytic formulas.
> ERROR: The variable n11 was referenced but not given a value.
> ERROR: The variable n12 was referenced but not given a value.
> ERROR: The variable n13 was referenced but not given a value.
> ERROR: The variable n14 was referenced but not given a value.
> ERROR: The variable n21 was referenced but not given a value.
> ERROR: The variable n22 was referenced but not given a value.
> ERROR: The variable n23 was referenced but not given a value.
> ERROR: The variable n24 was referenced but not given a value.
> ERROR: The variable n0 was referenced but not given a value.
> WARNING: Your program statements contain 9 symbols used but not given a
> value.
> WARNING: None of the data set variables are used in the program statements.
> NOTE: Initial value of parameter gamma is set randomly to 0.7504856474.
> NOTE: Initial value of parameter q1 is set randomly to 0.6170781951.
> NOTE: Initial value of parameter q2 is set randomly to 0.4308046808.
> NOTE: Initial value of parameter q3 is set randomly to 0.9153506276.
> NOTE: Initial value of parameter q4 is set randomly to 0.7203864049.
> WARNING: Your program statements cannot be executed completely.
> WARNING: In a total of 1 calls an error occurred during execution of the
> program statements. NLP attempted to recover by using a
> shorter step size.
> NOTE: View OUT.FORNLP.VIEW used (Total process time):
> real time 0.18 seconds
> cpu time 0.15 seconds
>
> NOTE: PROCEDURE NLP used (Total process time):
> real time 0.20 seconds
> cpu time 0.17 seconds
>
>
>
>
>
>
> LibName out 'C:\test';
> data out.forNLP / view=out.forNLP;
> set out.testset3; *the output data set from RANDMULTINOMIAL;
> replicate = _n_;
> run;
>
> %macro _overall(_itcnt=500);
> %do _i=1 %to &_itcnt;
>
> proc NLP data=out.forNLP;
> by replicate;
> *the rest;
>
>
> f1=.25;f2=.25;f3=.25;f4=.25;
> max h;
> decvar gamma q1 q2 q3 q4;
> bounds 0<=gamma<=1;
> bounds 0<=q1<=1;
> bounds 0<=q2<=1;
> bounds 0<=q3<=1;
> bounds 0<=q4<=1;
>
>
>
> h=n11*log(f1*gamma*q1)+n12*log(f2*gamma*q2)+n13*log(f3*gamma*q3)+n14*log
> (f4*gamma*q4)+n21*log(f1*q1*gammaf1*q1*q1*gamma)+n22*log(f2*q2*gamma
> f2*q2*q2*gamma)+n23*log(f3*q3*gammaf3*q3*q3*gamma)+n24*log(f4*q4*gamma
> f3*q4*q4*gamma)+n0*log((1gamma)+gamma*((1q1)*(1q1)*f1+(1q2)*(1q2)*f2+
> (1q3)*(1q3)*f3+(1q4)*(1q4)*f4));
> run;
> %end;
>
> %mend _overall;
>
> %_overall;
>
>
>
>
>
>
>
> On Wed, 9 Jan 2008 08:41:13 0600, data _null_, <datanull@GMAIL.COM> wrote:
>
> >I think you need to use the BY statement in PROC NLP.
> >
> >after you create the output data set from RANDMUNTINOMIAL use a data
> >step view to create a replicate variable then run proc nlp by
> >replicate;
> >
> >data work.forNLP / view=work.forNLP;
> > set work.x; *the output data set from RANDMULTINOMIAL;
> > replicate = _n_;
> > run;
> >
> >proc NLP data=work.forNLP;
> > by replicate;
> > *the rest;
> > run;
> >
> >
> >On Jan 9, 2008 8:20 AM, Randall Powers <powers_r@bls.gov> wrote:
> >> Thanks to all who have been replying. Maybe I haven't been clear enough
> as
> >> to what I'm trying to do. So I'll try to explain again.
> >>
> >> I have a function h that I'm using PROC NLP to maximize:
> >>
> >> h=n11*log(f1*gamma*q1)+n12*log(f2*gamma*q2)+n13*log(f3*gamma*q3)+n14*log
> >> (f4*gamma*q4)+n21*log(f1*q1*gammaf1*q1*q1*gamma)+n22*log(f2*q2*gamma
> >> f2*q2*q2*gamma)+n23*log(f3*q3*gammaf3*q3*q3*gamma)+n24*log(f4*q4*gamma
> >> f3*q4*q4*gamma)+n0*log((1gamma)+gamma*((1q1)*(1q1)*f1+(1q2)*(1q2)
> *f2+
> >> (1q3)*(1q3)*f3+(1q4)*(1q4)*f4));
> >>
> >> The decision variables are:
> >> gamma q1 q2 q3 q4
> >>
> >> I assign PROC NLP the values of f1,f2,f3,f4.
> >>
> >> I can also assign PROC NLP the values of n0, n11, n12, n13, n14, n21,
> n22,
> >> n23, n24. However, I want to run this nlp many (let's say P) times, so I
> am
> >> using PROC IML to generate a matrix of values:
> >>
> >> n0 n11 n12 n13 n14 n21 n22 n23 n24
> >> n0 n11 n12 n13 n14 n21 n22 n23 n24
> >> .
> >> .
> >> .
> >> n0 n11 n12 n13 n14 n21 n22 n23 n24 (Pth row)
> >>
> >> to use as input.
> >>
> >> Thus, each row of this matrix are the variables to a maximization
> problem
> >> that I wish to run P times. So I need a matrix of P rows and (in this
> case,
> >> though it may change depending on what I'm maximizing) 9 columns (I made
> my
> >> previous example the the list a 10x3 matrix, for sake of keeping it
> simple.)
> >>
> >> The values of n0, n11, n12, n13, n14, n21, n22, n23, n24 are determined
> >> using the proc iml statement below. For this example, we would get a 10x9
> >> matrix, where the values in each row would add up to 50, and the
> >> probability of randomly getting each of the values n0, n11, n12, n13,
> n14,
> >> n21, n22, n23, n24 would be 0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.2,
> >> respectively.
> >>
> >>
> >> proc iml;
> >> load module=RANDMULTINOMIAL;
> >> /* doc example */
> >> call randseed(1);
> >> prob = {0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.2};
> >> NumTrials = 50;
> >> N = 10;
> >> x = RANDMULTINOMIAL(N,NumTrials,prob);
> >> create out.testset2 from x ;
> >> append from x;
> >> print x;
> >> quit;
> >>
> >> Thus, my hope is to take the 10x9 (or 10x3 in my previous example, or
> >> Pxwhatever) and use it as an array of inputs, with each row being a
> >> separate collection of variables for a single PROC NLP run.
> >>
> >> Thanks again to whoever can make heads or tails of this and offer
> sensible
> >> ideas!
> >>
> >> On Tue, 8 Jan 2008 15:39:04 0600, data _null_, <datanull@GMAIL.COM>
> wrote:
> >>
> >> >I thought you wanted to run PROC NLP. What is the array for?
> >> >
> >> >On Jan 8, 2008 3:31 PM, Randall Powers <powers_r@bls.gov> wrote:
> >> >> Ok, that gives me a data set with 10 rows and 3 columns.
> >> >>
> >> >> Perhaps a dumb question, but how do I now get this dataset into a two
> >> >> dimemsional 10x3 array?
> >> >>
> >> >> Thanks again!
> >> >>
> >> >>
> >> >> On Tue, 8 Jan 2008 14:10:17 0600, data _null_, <datanull@GMAIL.COM>
> >> wrote:
> >> >>
> >> >> >Try the FROM clause
> >> >> >
> >> >> >create out.testset2 from x;
> >> >> >append from x;
> >> >> >
> >> >> >
> >> >> >
> >>
> >> >> >On Jan 8, 2008 1:26 PM, Randall Powers <powers_r@bls.gov> wrote:
> >> >> >> Thanks, Datanull.
> >> >> >>
> >> >> >> I was asking if there was a way to convert the matrix into an array
> >> >> >> directly within proc iml. If this is not possible, then I would go
> the
> >> >> >> route you suggest...
> >> >> >>
> >> >> >> Indeed, I had thought to do that and had gotten that far. Take the
> >> >> matrix,
> >> >> >> create a dataset (that's as far as I'd gotten), then assign it
> into an
> >> >> >> array.
> >> >> >>
> >> >> >> One kink I'm noticing (see example below)... This code produces a
> 10x3
> >> >> >> matrix where each row adds up to 50. However, when I use the create
> >> and
> >> >> >> append statements (and a subsequent data step to rid myself of all
> >> other
> >> >> >> variables), it creates a dataset that is 30x1, rather than the
> >> promised
> >> >> >> 10x3 dataset.
> >> >> >>
> >> >> >> So two questions:
> >> >> >>
> >> >> >> 1. How do I get it to create a 10x3 dataset which mimics my
> matrix, as
> >> >> >> promised?
> >> >> >>
> >> >> >> OR
> >> >> >>
> >> >> >> 2. How do I read the 30x1 dataset into my desired 10x3 array?
> >> >> >>
> >> >> >>
> >> >> >>
> >> >> >> proc iml;
> >> >> >> load module=RANDMULTINOMIAL;
> >> >> >> call randseed(1);
> >> >> >> prob = {0.3,0.6,0.1};
> >> >> >> NumTrials = 50;
> >> >> >> N = 10;
> >> >> >> x = RANDMULTINOMIAL(N,NumTrials,prob);
> >> >> >> create out.testset2 ;
> >> >> >> append;
> >> >> >> print x;
> >> >> >> quit;
> >> >> >>
> >> >> >> data out.testset3;
> >> >> >> set out.testset2 (keep=x);
> >> >> >> run;
> >> >> >>
> >> >> >>
> >> >> >> SAS proc iml output:
> >> >> >>
> >> >> >>
> >> >> >> 14 26 10
> >> >> >> 13 30 7
> >> >> >> 13 28 9
> >> >> >> 19 25 6
> >> >> >> 15 27 8
> >> >> >> 15 29 6
> >> >> >> 14 27 9
> >> >> >> 20 27 3
> >> >> >> 18 26 6
> >> >> >> 11 32 7
> >> >> >>
> >> >> >> SAS dataset out.testset3:
> >> >> >>
> >> >> >> 14
> >> >> >> 26
> >> >> >> 10
> >> >> >> 13
> >> >> >> 30
> >> >> >> 7
> >> >> >> 13
> >> >> >> 29
> >> >> >> 9
> >> >> >> .
> >> >> >> .
> >> >> >> .
> >> >> >> 11
> >> >> >> 32
> >> >> >> 7
> >> >> >>
> >> >> >> On Tue, 8 Jan 2008 10:53:23 0600, data _null_,
> <datanull@GMAIL.COM>
> >> >> wrote:
> >> >> >>
> >> >> >> >I think, you want to create a SAS data set from a matrix...
> >> >> >> >
> >> >> >> >Creating a SAS Data Set from a Matrix
> >> >> >> >SAS/IML software provides the ability to create a new SAS data set
> >> >> >> >from a matrix. Use the CREATE and APPEND statements to create a
> SAS
> >> >> >> >data set from a matrix, where the columns of the matrix become the
> >> >> >> >data set variables and the rows of the matrix become the
> >> observations.
> >> >> >> >Thus, an n matrix creates a SAS data set with m variables and n
> >> >> >>
> >> >> >> >observations. The CREATE statement opens the new SAS data set for
> >> both
> >> >> >> >input and output, and the APPEND statement writes to (outputs to)
> the
> >> >> >> >data set.
> >> >> >> >
> >> >> >> >Then use that data set as input to PROC NLP.
> >> >> >> >
> >> >> >> >On Jan 8, 2008 10:30 AM, Randall Powers <powers_r@bls.gov> wrote:
> >> >> >> >> Sorry this is rambling and longwinded. Any input anybody can
> give
> >> >> would
> >> >> >> be
> >> >> >> >> helpful! If what I'm saying/asking doesn't make sense, it's
> >> probably
> >> >> >> >> because of my newbieness...I'd be happy to clarify.
> >> >> >> >>
> >> >> >> >> So here is what I'm doing.
> >> >> >> >>
> >> >> >> >> I'm using proc iml to create a matrix of values. The number of
> >> columns
> >> >> >> and
> >> >> >> >> rows may vary, but for arguments sake, let's say 50 rows and 5
> >> >> columns.
> >> >> >> I'm
> >> >> >> >> using RANDMULTIMOMIAL to create matrix X, which is 50x5. What
> I'd
> >> >> like to
> >> >> >> >> do is somehow convert that matrix into a 50x5 array (50 rows, 5
> >> >> columns),
> >> >> >> >> which I would then use as input in a proc nlp.
> >> >> >> >>
> >> >> >> >> For example:
> >> >> >> >>
> >> >> >> >> 5 4 6 2 5
> >> >> >> >> 4 5 4 4 5
> >> >> >> >> 3 7 3 6 3
> >> >> >> >> .........
> >> >> >> >> .........
> >> >> >> >> etc
> >> >> >> >>
> >> >> >> >> Does this seem possible? Can it all be done in the proc iml, or
> do
> >> I
> >> >> have
> >> >> >> >> to break it up and do it outside of proc iml?
> >> >> >> >>
> >> >> >> >> One thing I did try was to create an output file which I'm
> thinking
> >> >> could
> >> >> >> >> then be used to read the values of X into an array in a data
> step.
> >> The
> >> >> >> >> output file created from proc iml assigns each value of the
> matrix
> >> a
> >> >> >> >> separate observation. Thus, the example below gives a SAS data
> set
> >> >> with:
> >> >> >> >>
> >> >> >> >> OBS X
> >> >> >> >> 1 5
> >> >> >> >> 2 4
> >> >> >> >> 3 6
> >> >> >> >> 4 2
> >> >> >> >> 5 5
> >> >> >> >> 6 4
> >> >> >> >> 7 5
> >> >> >> >> 8 4
> >> >> >> >> 9 4
> >> >> >> >> 10 5
> >> >> >> >> etc. (in this example, I would have 50x5=250 obs)
> >> >> >> >>
> >> >> >> >> How would I then capture the value of X for the first five
> >> >> observations
> >> >> >> >> into an array, the second five values into another array, the
> third
> >> >> five
> >> >> >> >> values into a third array, etc. so that I would have 50 arrays
> of
> >> five
> >> >> >> >> values (or a multidimensional array of size {50,5} containing
> all
> >> of
> >> >> the
> >> >> >> >> values? I would then send this array to my proc nlp, where it
> >> would be
> >> >> >> used.
> >> >> >> >>
> >> >> >>
> >> >>
> >>
>
