thnak you for your reply. I don't understant how do i find 'stepwise
selection' (except logistic regression) in enterprise miner GUI. there
are few more problems I have during the model developement which are:
A. for this problem I have made a sample of 150,000 records with 80
variables and saved it on a hard disk. Now the problem is when I build
the tree then it only shows two 1 node with two leaves ( while responce
has (34,000 (0) and 80,000 (1) and some missing data). I don't
understant how to create tree for this model.
B. With the similar dataset, I also tried to build Neural network and
it doesnot shows any result.
I am wondering, I may be failed to develope such a model. Suggest some
way to get out from this point.
Sigurd Hermansen wrote:
> Sounds like a good opportunity for you to analyze what is happening and =
> find a good solution....
> I'd look first at obvious bottlenecks. Is EM trying to transfer millions =
> of rows in a table on a server to another node on a network. Network =
> transfer speeds will slow things down. Are you following the dubious =
> practice of searching very large numbers of predictors for a relevant =
> model? (If so, search the SAS-L Archives for postings on 'step-wise =
> selection'.) Moving large volumes of data takes time.
> A good 150,000 row sample of a larger database should give you good =
> estimates of all but very small proportions. The key word is 'good'. If =
> you know little about your data source and choose a sample with an =
> underrepresentation of an important subgroup, you could minimize the =
> importance of the subgroup and find a good fit for a bad model.
> EM and all other automated data mining programs may facilitate modelling =
> but, as with all statistical packages, also make it too easy to go =
> astray. Don't expect easy answers.
> Remember to train models on one sample, test models on another =
> independent sample, and, after selecting a specific model, evaluate its =
> performance on one or more samples not used in training and testing. You =
> have enough data to implement that strategy.
> Best wishes for exciting challenges and success in your academic work.
> From: email@example.com on behalf of firstname.lastname@example.org
> Sent: Tue 9/5/2006 7:51 AM
> To: email@example.com
> Subject: Enterprise miner problem
> I am a student and working on a project. I have enterprise miner and
> working on a on a dataset (there is millions of records) which is on
> server. The problem is when i tried to connect data with in "input data
> node", the computer and SAS software become very slow. I had wait about
> more than 30 minutes to get responce. I have taken a sample from the
> same dataset (more than 150,000 records) and saved on local drive and
> it works properly. If this problem persis, is there any solution
> without using enterprise miner GUI?