LISTSERV at the University of Georgia
Menubar Imagemap
Home Browse Manage Request Manuals Register
Previous messageNext messagePrevious in topicNext in topicPrevious by same authorNext by same authorPrevious page (June 2005, week 1)Back to main SAS-L pageJoin or leave SAS-L (or change settings)ReplyPost a new messageSearchProportional fontNon-proportional font
Date:         Fri, 3 Jun 2005 14:08:24 -0500
Reply-To:     schmitta@UWM.EDU
Sender:       "SAS(r) Discussion" <SAS-L@LISTSERV.UGA.EDU>
From:         "Thomas A. Schmitt" <schmitta@UWM.EDU>
Subject:      Design/Analysis Questions #2
Comments: To: cassell.david@EPAMAIL.EPA.GOV
In-Reply-To:  <OF25AB2D83.36E6D64A-ON88257014.00785AC3-88257014.0078E563@epamail.epa.gov>
Content-Type: text/plain; charset=ISO-8859-1

Hello All,

Here are some more details on my study. There are nine experimental conditions and one control condition and examinee responses within each of these conditions are simulated using the 1-PL and 3-PL models in IRT. Each examinee (with a known theta) is given a 30-item test that is completed “normally” using the probabilities generated from the models (this is the control condition). The simulated CAT also included nine experimental conditions pertaining to three different completion methods and the three different numbers of items these methods were completed in. Thus, and examinee either guesses and does not complete the last 3, 6, and 9 test items. For the guessing method examinees had a 0%, 25% or 33% chance of answering an item correctly. These probabilities replaced the model generated probabilities and were intended to simulate speededness in the sense that examinees would have to hurry near the end of a test to complete items. Thus, one control condition (normally completed 30-item test) and 9 experimental conditions (3, 6, and 9 test items; 33%, 25%, and 0%) were considered. My hypothesis is that as the number of items guessed on increases (moving from 3 to 9 items) and the probability of getting these items corrected decreased (moving from 33% to 0%) the examinees thetas will be “significantly” underestimated.

I simulated each of the 200 examinees 100 times for the 9 experimental conditions and then combined across replications.

Since this is a 3x3 within-subjects design I wanted to analyze the 9 experimental conditions appropriately in SAS. I’m still trying to decide how to fit the control condition into the picture since it does not have the 2 factors. I would like to test the main effects and the interactions in SAS. One of my concerns with this study has been statistical significance. Do I report this or do I just simply use effects. Because of the small error and large sample size the results will be very significant.

Since I have calculated Bias and RMSE for each of the 200 examinees one thought here was to take a mean of the Bias and RMSE for each of the 200 examinees in each of the 9 conditions along with the control condition and create a histogram with the mean of the RMSEs and mean Biases. This would give me a mean effect of sorts and allows me to see the increase in the underestimate of theta. Does this seem reasonable even though the conditions are not separated?

Any thoughts on this design and data analyses and summarization and how to do the 3x3 within-subjects design in SAS testing the main effects and interactions would be appreciated. Anyway, I hope this clarifies somewhat my study.

Tom


Back to: Top of message | Previous page | Main SAS-L page