Markov Chain

[LFC]

hey all.....was hoping someone could explain the following to me...

The number of future periods spent in current state i, prior to a change in state, follows a Geometric Distribution with beta=failure/success where failure is defined as the Pr[no change] and success as Pr[change].

Therefore, the expected future periods spent in state i until change is equal to beta.

BUT....the mean time between a return to state i = 1/{stationary distribution of i}.

i think i'm having a mental block..but i i cant see how these two are different.

thanks in advance.

[Gandalf]

From your description, here's what's different:

Consider a system with states 1,2,3.

For the expected time to leave state 1, you are only considering time in state 1: how long do you remain there until you go somewhere else.

For expected time until you return to state 1, you are definitely counting time outside state 1. Thus, if it is very hard to leave 2 and 3, then there will be a long time before returns to state 1. (You would have to check the text or study guide to get the exact definition: I'm not sure whether time within 1 is also included, so that it is average time between arrivals in state 1. But it definitely includes time outside 1.)

[LFC]

thanks gandalf. now for what u wrote in the 2nd para...for the "expected time until u return to state 1"...is the formula for this:
1/{stationary dist of state 1}?

mahler's notes do make reference to what u were syaing in brackets...but thats regarding the "expected number of time periods in state j, if it starts in state i". this is calculated as:

{prob. u vist state j, if u start in state i} * {expected periods spent in state j, if u start in period j}.

and if i=j, then we add one.



From your description, here's what's different:

Consider a system with states 1,2,3.

For the expected time to leave state 1, you are only considering time in state 1: how long do you remain there until you go somewhere else.

For expected time until you return to state 1, you are definitely counting time outside state 1. Thus, if it is very hard to leave 2 and 3, then there will be a long time before returns to state 1. (You would have to check the text or study guide to get the exact definition: I'm not sure whether time within 1 is also included, so that it is average time between arrivals in state 1. But it definitely includes time outside 1.)

[Gandalf]

Sorry, not taking the exam. I could figure out the general meaning just from looking at the words you wrote, but formula :shake:

[Ducklin]

From Gandalf: I'm not sure whether time within 1 is also included, so that it is average time between arrivals in state 1. But it definitely includes time outside 1.

It does not include time withing state 1. I study the course but I do not have the notes infront of me. Will have to look it up to be able to give you the correct answer, don't want to mislead you!

:roll: