John, As I understand your description, the pre-post design you have allows for comparing dependent proportions via McNemar's test if you have a binary indicator, and a more general test of marginal homogeneity if your indicator is polytomous. Your approach of creating a series of binary indicators is probably not the best approach. If you are willing to treat your Likert items as ordinal, then StatXact provides a test of marginal homogeneity for a kxk table. For some discussion, see Agresti's Categorical Data Analysis book, Haberman's old two-volume book on analyzing qualitative data, or StatXact documentation.

Anthony Babinec

-----Original Message----- From: SPSSX(r) Discussion [mailto:SPSSX-L@LISTSERV.UGA.EDU] On Behalf Of John Norton Sent: Wednesday, June 29, 2005 3:54 PM To: SPSSX-L@LISTSERV.UGA.EDU Subject: McNemar Test and Study Design

Hi List,

We've been asked by a reviewer to provide further explanations about why we chose to use a McNemar test to determine whether the marginal proportions in a 2X2 table were statistically different, and I am hoping some on this list may be able to provide some guidance.

The study employed a pre/post design where patients were surveyed at intake and then again after their consultations. So, the same people were surveyed twice, once at intake and again after the consultation (which is in effect the "treatment" we're studying). As I understand it, the McNemar test is appropriate for this design.

The response alternatives to the survey questions used a 5 point Likert format, with responses ranging from Strongly Disagree to Strongly Agree. So, to use the McNemar, each response alternative to each question was transformed into a binary variable, such that a "Strongly Agree" answer to Q4 on the pretest meant that the variable Q4pre_e = 1, while Q4pre_a through Q4pre_d all were assigned a zero. Likewise, on the post test, if the respondent answered "Neither" then Q4post_c = 1, while the remaining 4 "post" variables were assigned a value of 0. McNemar tests were executed on all 5 pairs of pre/post variables; none of them were significant.

The reviewer raised the question, "although roughly 1 in 5 patients answered "Strongly Agree" both before and after the consultation, were they the same individuals?" The immediate answer is, well, yes, as is defined in the design of the study. But if the reviewer is actually asking whether the 19% answering "Strongly Agree" at intake are the same individuals composing the 21% answering "Strongly Agree" on the post-test, I am left wondering whether this is a moot concern, or whether this may confound the results of the McNemar. Can anyone help clarify?

Many thanks,

John Norton Biostatistician Oncology Institute Loyola University Medical Center

(708) 327-3095 jnorton@lumc.edu

"Everything that can be counted isn't worth counting, and everything that is worth counting isn't always countable." - Einstein