As I understand your description, the pre-post design you
have allows for comparing dependent proportions via
McNemar's test if you have a binary indicator, and a more
general test of marginal homogeneity if your indicator
is polytomous. Your approach of creating a series of
binary indicators is probably not the best approach.
If you are willing to treat your Likert items as ordinal,
then StatXact provides a test of marginal homogeneity
for a kxk table. For some discussion, see Agresti's
Categorical Data Analysis book, Haberman's old
two-volume book on analyzing qualitative data,
or StatXact documentation.
From: SPSSX(r) Discussion [mailto:SPSSX-L@LISTSERV.UGA.EDU] On Behalf Of
Sent: Wednesday, June 29, 2005 3:54 PM
Subject: McNemar Test and Study Design
We've been asked by a reviewer to provide further explanations about
why we chose to use a McNemar test to determine whether the marginal
proportions in a 2X2 table were statistically different, and I am hoping
some on this list may be able to provide some guidance.
The study employed a pre/post design where patients were surveyed at
intake and then again after their consultations. So, the same people
were surveyed twice, once at intake and again after the consultation
(which is in effect the "treatment" we're studying). As I understand
it, the McNemar test is appropriate for this design.
The response alternatives to the survey questions used a 5 point Likert
format, with responses ranging from Strongly Disagree to Strongly Agree.
So, to use the McNemar, each response alternative to each question was
transformed into a binary variable, such that a "Strongly Agree" answer
to Q4 on the pretest meant that the variable Q4pre_e = 1, while Q4pre_a
through Q4pre_d all were assigned a zero. Likewise, on the post test,
if the respondent answered "Neither" then Q4post_c = 1, while the
remaining 4 "post" variables were assigned a value of 0. McNemar tests
were executed on all 5 pairs of pre/post variables; none of them were
The reviewer raised the question, "although roughly 1 in 5 patients
answered "Strongly Agree" both before and after the consultation, were
they the same individuals?" The immediate answer is, well, yes, as is
defined in the design of the study. But if the reviewer is actually
asking whether the 19% answering "Strongly Agree" at intake are the same
individuals composing the 21% answering "Strongly Agree" on the
post-test, I am left wondering whether this is a moot concern, or
whether this may confound the results of the McNemar. Can anyone help
Loyola University Medical Center
"Everything that can be counted isn't worth counting,
and everything that is worth counting isn't always countable."