Date:         Fri, 20 Jul 2007 13:16:21 -0500
Reply-To:     Linda Case <lcase@autumngoldconsulting.com>
Sender:       "SPSSX(r) Discussion" <SPSSX-L@LISTSERV.UGA.EDU>
From:         Linda Case <lcase@autumngoldconsulting.com>
Subject:      Selection of Appropriate Tests of Association (long)
In-Reply-To:  <46A08841.4269.00D8.0@jsi.com>
Content-Type: text/plain; charset="us-ascii"

Greetings -

I have a question that may not have an answer, as I am not sure that the
comparisons that are being requested are even possible.  On the other hand,
I also fear that I may be missing something very simple and basic!  Here is
the problem:

I have two sets of data. The first is a very large consumer test (CT) that
asked a very large group of participants to evaluate six products after
using them at home. Among many other things, a series of 9 descriptive
questions about each product were included in this study. A 10th relevant
question is an "intent to purchase" query.  Each participant assessed one
product.

The second set of data is a small pilot study of 21 participants. None of
the participants in this small pilot study were tested in the big CT (i.e.
completely different group of people).  This is a repeated measures design
in which the same six products are evaluated by each participant and each
product was assessed using three separate assessment tools (i.e. ways of
viewing/interacting with the product).  This test took place in a lab, with
no in-home use.  The same 9 descriptive questions and the 10th intent to
purchase questions were used to evaluate.

My client's goal is to assess the degree of association between answers
provided in the small study with those provided for the same products in the
large study. They wish to see how predictive the three assessment tools are
of actual answers after actual use of the product.  They asked to start with
simple tests of association (correlation) between each question for each
product using the assessment tools in the small study and the answers to the
same questions from the large CT.  The ultimate goal (of the client) is to
conduct either multiple linear regression or logistic regression on the 9
descriptive questions (IV) to determine their ability to predict the outcome
of the intent to buy question (DV).  Of course this would be quite possible
within the big study. However, they are requesting that this is done with
one of the "Assessment tools" used in the small study, NOT with the data
collected from the large CT.  In other words, the IVs would come from the
small study and the DV from the large study.  I do not see how this is
possible because the participants are not the same in each study (not to
mention the extremely small sample size in the pilot study, but I think that
is a secondary issue, since I do not see how I can run any tests of
association between two unrelated samples).

My client is first interested in finding out how well the three assessment
tools in the small study agree with answers from the large study. Although
they want correlation, I could think of no way to provide that, so instead
have treated the large data set as "the population" and used aggregate to
create new variables for each question and product that are the mean for
each.  I used these as the population mean (seeing that these really do
represent whether or not people would purchase the product after using it),
and have conducted a series of single-sample t-tests for each product and
question against its corresponding "population" mean from the large CT.
These do provide information regarding whether or not the means for each of
the assessment tools differ significantly (or not) from the large CT mean.
This is where I am stuck.  I cannot run any type of correlation tests
because the cases (subjects) in each group are completely different people.
My client is very adamant about wanting some test of association.  Is there
more that I can do with this design to provide the answers that my client is
requesting? (Again, if I am missing something that is very simple and
obvious, I apologize ahead of time!)

Thanks for your help in advance (since I know this is long, please feel
welcome to reply off the list if you prefer).

Linda Case

Linda P. Case
AutumnGold Consulting
(217) 586-4864
www.autumngoldconsulting.com
lcase@autumngoldconsulting.com or lcase@uiuc.edu