Date:         Thu, 18 Oct 2007 10:32:21 -0500
Reply-To:     Warren Schlechte <Warren.Schlechte@TPWD.STATE.TX.US>
Sender:       "SAS(r) Discussion" <SAS-L@LISTSERV.UGA.EDU>
From:         Warren Schlechte <Warren.Schlechte@TPWD.STATE.TX.US>
Subject:      Re: Method to distinguish different periods in time series data
Content-Type: text/plain; charset="us-ascii"

Following up on the Regime-Shift idea, that suggests quality-control and
control charts could be a useful arena as well.  There are both
parametric and nonparametric constructs for cumsum and exponentially
weighted moving averages (EWMA). 

Warren Schlechte

-----Original Message-----
From: Samuel Croker [mailto:samuel.croker@GMAIL.COM] 
Sent: Thursday, October 18, 2007 9:21 AM
Subject: Re: Method to distinguish different periods in time series data

If you are pretty sure that you know when the change happened, then an
intervention analysis is the way to go.  I think that you should be
able to find ample information on how this is done in the SAS/ETS
documentation, particularly in PROC ARIMA, although only one of your
variables can be the response at one time.

As to finding the change, the following link has a nice comparison of
some of the methods of what they call "regime shift" detection -
finding an unknown intervention in the data.

www.beringclimate.noaa.gov/regimes/rodionov_overview.pdf

It may spur some research ideas for you.  There is a large amount of
literature on this topic. I experimented with wavelet decomposition
for anomaly detection and it was enlightening, but not particularly
helpful for the problem that I was working on.  "anomaly detection" is
another good search phrase since it goes well beyond outlier
detection, which can be problematic in this setting.


Sam


On 10/18/07, Warren Schlechte <Warren.Schlechte@tpwd.state.tx.us> wrote:
> Seems to me a good starting point would be a loess fit.
>
> Final step seems like intervention analysis, although I believe there
> you should have a known intervention.
>
> Hope these thoughts help.
>
> Warren Schlechte
>
> -----Original Message-----
> From: kansaskannan@GMAIL.COM [mailto:kansaskannan@GMAIL.COM]
> Sent: Wednesday, October 17, 2007 8:28 PM
> Subject: Re: Method to distinguish different periods in time series
data
>
> The variables are of unknown importance; I did make my initial
> groupings based on graphs, but wondered if there was a better way.
>
> Thanks, Sam,  for taking the time to respond.
>
>
> On Oct 15, 10:02 pm, samuel.cro...@GMAIL.COM (Samuel Croker) wrote:
> > I think that some good graphing can say a whole lot, and does not
get
> > into the potentially problematic area of doing statistical inference
> > without thinking hard about it - always a dangerous area.  Graphs
can
> > be helpful in getting your ideas together about next steps, and if
> > done ethically and precisely they can be great descriptive tools to
> > support your other analysis.
> >
> > Now you mentioned that you have several variables.  Do you have a
> > single target variable and the others potentially affect this one,
or
> > are all of the variables of equal or unknown importance?
> >
> > Sam
> >
> > On 10/15/07, kansaskan...@gmail.com <kansaskan...@gmail.com> wrote:
> >
> >
> >
> >
> >
> > > Thank you Sam. You are right, I am doing preliminary work to
> determine
> > > if there is indeed a shift in the phenomenon (home foreclosures) I
> am
> > > studying.  I would like to find out if the number of foreclosures
> has
> > > changed significantly, and specifically when this change happened.
> I
> > > did do a t-test to compare two arbitrary periods, but wondered if
a
> > > different choice of periods would not also give me a significant
> > > difference.  I could repeat the process several times for
different
> > > pairs of arbitrary periods.   I wondered if there were any SAS
> > > procedures - hence the post in this forum -  that I could use to
> > > determine (a) whether there is a significant change; (b) when the
> > > change happened; (c) whether there are more than two 'periods'
that
> > > can be identified.
> >
> > > 11:45 am, samuel.cro...@GMAIL.COM (Samuel Croker) wrote:
> > > > A little more info might be helpful.  It does not sound as if
you
> are
> > > > trying to model yet, but conducting some preliminary data
> analysis.
> > > > Is this right?  Are you trying to verify seasonality or are you
> > > > looking for other types of shifts in the data?
> >
> > > > Sam
> >
> > > > On 10/15/07, kansaskan...@gmail.com <kansaskan...@gmail.com>
> wrote:
> >
> > > > > I have a time series dataset (monthly data; 220 observations).
I
> need
> > > > > to determine if I can distinguish different periods, based on
> (a)
> > > > > changes in one variable; (b) changes in two or three variables
> taken
> > > > > together.
> >
> > > > > I would appreciate any pointers in simple terms.
> >
> > > > > Thank you.
> >
> > > > --
> > > > Samuel T. Croker
> > > > Lexington, SC & Bethesda, MD
> >
> > --
> > Samuel T. Croker
> > Lexington, SC & Bethesda, MD- Hide quoted text -
> >
> > - Show quoted text -
>


--
Samuel T. Croker
Lexington, SC & Bethesda, MD