|Date: ||Mon, 24 Nov 2003 09:24:15 -0500|
|Reply-To: ||"DePuy, Venita" <depuy001@DCRI.DUKE.EDU>|
|Sender: ||"SAS(r) Discussion" <SAS-L@LISTSERV.UGA.EDU>|
|From: ||"DePuy, Venita" <depuy001@DCRI.DUKE.EDU>|
|Subject: ||Re: Test meets Control -- ad effectiveness study|
Hi Dave -
not too sure about your selectivity being statistically justifiable . . .
would a better method be to do a regression on controls & trt groups
seperately, then see if there is a significant difference between the slopes
of the two lines?
i.e. if the economy is going to pot - or for whatever reason, sales are
decreasing - perhaps better advertising would reflect a smaller overall drop
in sales per person.
> From: Dave Mattingly[SMTP:carpedmnow@MSN.COM]
> Reply To: Dave Mattingly
> Sent: Sunday, November 23, 2003 2:53 PM
> To: SAS-L@LISTSERV.UGA.EDU
> Subject: Test meets Control -- ad effectiveness study
> I've got a frustrating little problem that I hope someone out there can
> help with.
> The company I work for has placed some ads in certain ZIP codes and has a
> group of other ZIP codes as the control group. The idea being to assess
> the ads' effectiveness at increasing some metric (let's use sales/person).
> Whoever set up these test and control groups made sure the overall
> sales/person for about 15 weeks before the ad-period was equivalent
> the test and control.
> The problem is that if you look at sales/person for EACH WEEK, there is a
> downward trend. For instance, the ratio of Test sales/person to Control
> sales/person starts at about 1.24, and after many peaks and valleys (i.e.,
> it's NOT MONOTONICALLY decreasing), the ratio drops to as low as 0.84 just
> before the ads hit.
> PROPOSED SOLUTION:
> In order to get a good comparison of sales/person after the ads hit, we
> need the Test to look more like the Control in the pre-ad period. My idea
> was to choose some ZIP codes from among the Test group (which has about 3
> times as many ZIPs as the Control) so that this ratio is flat and has less
> spread about 1 for the pre-ad period. I would choose the ZIPs by summing
> across weeks -- the squared deviations of each Test ZIP code sales/person
> from the overall Control mean:
> sigma^2 = sum(i=1 to n) of [Zt(i) - Zcbar(i)]
> where n=number of weeks AND Zt=individual Test ZIP code AND Zcbar=mean of
> all Control ZIPs.
> Then I would choose the ZIPs which had the lowest sigma^2. (Since I have
> little choice, I'm willing to disregard the likelihood that the chosen
> ZIPs follow a different distribution than the Control ZIPs.)
> When I take the ZIPs with the lowest sigma^2, the ratio follows the same
> trend -- except that the range is HIGHER (2.4 down to 1.8)!! When I take
> the ZIPs with the highest sigma^2, the ratio follows the same trend --
> except that the range is LOWER (0.6 down to 0.2)!! When I take the middle
> ZIPs, the ratio follows the original trend and approximate range.
> How is this possible!?!? I've checked my computations at every step, and
> checked for things like missing values.
> Please help. I tearing my hair out here! Thanks in advance,
> D. Mattingly