Date: Thu, 18 Oct 2007 10:32:21 0500
ReplyTo: Warren Schlechte <Warren.Schlechte@TPWD.STATE.TX.US>
Sender: "SAS(r) Discussion" <SASL@LISTSERV.UGA.EDU>
From: Warren Schlechte <Warren.Schlechte@TPWD.STATE.TX.US>
Subject: Re: Method to distinguish different periods in time series data
ContentType: text/plain; charset="usascii"
Following up on the RegimeShift idea, that suggests qualitycontrol and
control charts could be a useful arena as well. There are both
parametric and nonparametric constructs for cumsum and exponentially
weighted moving averages (EWMA).
Warren Schlechte
Original Message
From: Samuel Croker [mailto:samuel.croker@GMAIL.COM]
Sent: Thursday, October 18, 2007 9:21 AM
Subject: Re: Method to distinguish different periods in time series data
If you are pretty sure that you know when the change happened, then an
intervention analysis is the way to go. I think that you should be
able to find ample information on how this is done in the SAS/ETS
documentation, particularly in PROC ARIMA, although only one of your
variables can be the response at one time.
As to finding the change, the following link has a nice comparison of
some of the methods of what they call "regime shift" detection 
finding an unknown intervention in the data.
www.beringclimate.noaa.gov/regimes/rodionov_overview.pdf
It may spur some research ideas for you. There is a large amount of
literature on this topic. I experimented with wavelet decomposition
for anomaly detection and it was enlightening, but not particularly
helpful for the problem that I was working on. "anomaly detection" is
another good search phrase since it goes well beyond outlier
detection, which can be problematic in this setting.
Sam
On 10/18/07, Warren Schlechte <Warren.Schlechte@tpwd.state.tx.us> wrote:
> Seems to me a good starting point would be a loess fit.
>
> Final step seems like intervention analysis, although I believe there
> you should have a known intervention.
>
> Hope these thoughts help.
>
> Warren Schlechte
>
> Original Message
> From: kansaskannan@GMAIL.COM [mailto:kansaskannan@GMAIL.COM]
> Sent: Wednesday, October 17, 2007 8:28 PM
> Subject: Re: Method to distinguish different periods in time series
data
>
> The variables are of unknown importance; I did make my initial
> groupings based on graphs, but wondered if there was a better way.
>
> Thanks, Sam, for taking the time to respond.
>
>
> On Oct 15, 10:02 pm, samuel.cro...@GMAIL.COM (Samuel Croker) wrote:
> > I think that some good graphing can say a whole lot, and does not
get
> > into the potentially problematic area of doing statistical inference
> > without thinking hard about it  always a dangerous area. Graphs
can
> > be helpful in getting your ideas together about next steps, and if
> > done ethically and precisely they can be great descriptive tools to
> > support your other analysis.
> >
> > Now you mentioned that you have several variables. Do you have a
> > single target variable and the others potentially affect this one,
or
> > are all of the variables of equal or unknown importance?
> >
> > Sam
> >
> > On 10/15/07, kansaskan...@gmail.com <kansaskan...@gmail.com> wrote:
> >
> >
> >
> >
> >
> > > Thank you Sam. You are right, I am doing preliminary work to
> determine
> > > if there is indeed a shift in the phenomenon (home foreclosures) I
> am
> > > studying. I would like to find out if the number of foreclosures
> has
> > > changed significantly, and specifically when this change happened.
> I
> > > did do a ttest to compare two arbitrary periods, but wondered if
a
> > > different choice of periods would not also give me a significant
> > > difference. I could repeat the process several times for
different
> > > pairs of arbitrary periods. I wondered if there were any SAS
> > > procedures  hence the post in this forum  that I could use to
> > > determine (a) whether there is a significant change; (b) when the
> > > change happened; (c) whether there are more than two 'periods'
that
> > > can be identified.
> >
> > > 11:45 am, samuel.cro...@GMAIL.COM (Samuel Croker) wrote:
> > > > A little more info might be helpful. It does not sound as if
you
> are
> > > > trying to model yet, but conducting some preliminary data
> analysis.
> > > > Is this right? Are you trying to verify seasonality or are you
> > > > looking for other types of shifts in the data?
> >
> > > > Sam
> >
> > > > On 10/15/07, kansaskan...@gmail.com <kansaskan...@gmail.com>
> wrote:
> >
> > > > > I have a time series dataset (monthly data; 220 observations).
I
> need
> > > > > to determine if I can distinguish different periods, based on
> (a)
> > > > > changes in one variable; (b) changes in two or three variables
> taken
> > > > > together.
> >
> > > > > I would appreciate any pointers in simple terms.
> >
> > > > > Thank you.
> >
> > > > 
> > > > Samuel T. Croker
> > > > Lexington, SC & Bethesda, MD
> >
> > 
> > Samuel T. Croker
> > Lexington, SC & Bethesda, MD Hide quoted text 
> >
> >  Show quoted text 
>

Samuel T. Croker
Lexington, SC & Bethesda, MD
