Date: Thu, 29 Sep 2005 08:36:40 -0400
Reply-To: Art@DrKendall.org
Sender: "SPSSX(r) Discussion" <SPSSX-L@LISTSERV.UGA.EDU>
From: Art Kendall <Art@DrKendall.org>
Organization: Social Research Consultants
Subject: Re: complex samples
In-Reply-To: <001901c5c4da$b837b3a0$5e64a8c0@russell>
Content-Type: text/plain; charset=us-ascii; format=flowed
The wider estimates I was referring to were for POP totals, means,
percentages, etc. As long as you use the correct weights, the point
estimates will be right. The interval estimates will be off. Of
course, if the proportion of non-response is not the same in all cells,
you need to further adjust the weights.
One big danger with things like crosstabs, is talking about observed
percentages as if they were different when the difference is readily
attributed to the vagaries of sampling. It is crucial that some test
find the difference to be inconsistent with sampling error before
reporting them as different. For any interpretation of results in terms
of policy, practice, etc., comparisons of point estimates that do not
come up as different should be treated as if the were the same. For
example, if 25% of the schools in the South use red pencils on official
forms, and 10% of the schools in the West use red pencils on official
forms, the difference in those percentages may very well be consistent
with what one might observe simply due to sampling error. If the point
estimate differences are statistically significant, using the error
terms as if you had a simple random sample (i.e., ignoring the degree to
which the stratification variables account for some of the variance),
you will find fewer differences as significant. You will be losing
power. But those differences will have plausibility. Of course, report
that the error estimates are inflated.
If this is a one time study, you might be able to find another NGO,
university, etc., that has the complex sampling module. Run all of
your runs on your base SPSS using the weights. Once everything is
ready, send the system file to the other agency.
Check the SPSS site. You might also be able to a trial version that you
should be able to use in 30 days if everything is ready to go. Also
check whether the educational price is that high and whether the agency
qualifies as educational.
Art
Art@DrKendall.org
Social Research Consultants
University Park, MD USA Inside the Washington, DC beltway.
(301) 864-5570
russell wrote:
>Art,
>
>Thank you for the response. I believe that their researchers can
>probably live with wider estimates, even though the results of the study
>are not yet available. Furthermore, most of the reported stats will be
>in the form of descriptive tables (cross tabs mostly). They are now only
>beginning to tackle inferential research etc. Russell
>
>-----Original Message-----
>From: Art Kendall [mailto:Art@DrKendall.org]
>Sent: 28 September 2005 04:31
>To: russell
>Cc: SPSSX-L@LISTSERV.UGA.EDU
>Subject: Re: complex samples
>
>If you do not use the complex samples module, given that all of the
>design factors are stratifications (fixed effects), the error estimates
>(confidence intervals) for the whole pop will be wider than necessary.
>If the obtained intervals using the base are sufficiently narrow that
>you can live with them, then you might forego using the complex
>samples. If you intend to compare and contrast sets of cells, then you
>would be better off using the smaller error terms from complex samples.
>
>"The size of the sample is partly
>based on cost considerations, logistics etc."
>
>How are you gathering your data? By interview of by paper-and-pencil
>reports by the schools?
>
>Keep in mind that the total cost of a survey is NOT a simple direct
>effect of the sample size, especially in phone or mail surveys. A great
>deal of the cost is in instrument development, results reporting, etc.
>
>
>"The NGO claims that dividing the
>schools into strata means that the sample is more likely to be
>representative as you can ensure that each of the strata is represented
>proportionally within the sample."
>
>Proportional representation is important for ease in calculation of
>precision of pop estimates. It also helps plausibility of the design.
>For comparing and contrasting strata etc., equal cell sizes yield
>smaller error estimates.
>
>Art
>Art@DrKendall.org
>Social Research Consultants
>University Park, MD USA Inside the Washington, DC beltway.
>(301) 864-5570
>
>
>
>russell wrote:
>
>
>
>>Hi there,
>>
>>I am asking this question on behalf of an NGO that surveys primary
>>schools. They look at service delivery issues, leakage of funding,
>>corruption etc. From approximately 5000 primary schools, the sampling
>>frame is ten per cent (or 500 schools). The size of the sample is
>>
>>
>partly
>
>
>>based on cost considerations, logistics etc. The 500 schools were
>>selected through a random, 2 stage probability proportionate to size
>>selection process based on the number of schools. The schools were
>>divided according to regions: Northern region, Central Region and
>>Southern region. Then the schools in the region were further divided
>>into rural and urban schools. Finally, the schools were proportionally
>>selected through random sampling. The NGO claims that dividing the
>>schools into strata means that the sample is more likely to be
>>representative as you can ensure that each of the strata is represented
>>proportionally within the sample.
>>
>>Does analysis of this data require the complex samples module in SPSS?
>>
>>Any thoughts are appreciated,
>>Russell
>>
>>
>>
>>
>>
>>
>
>
>
>
>
| 
