Understanding Markov Chains and Poisson Processes in Depth
School
University of New South Wales**We aren't endorsed by this school
Course
COMM 1140
Subject
Statistics
Date
Dec 12, 2024
Pages
7
Uploaded by CommodoreFalconPerson558
Week 8: Supplemental examplesQuestion 1:FindPHC(x, y) in the H-C-D model.Question 2:Write down the K-backward equation for non-homegeneous Poisson process.Question 3:Recall in Q2 Week 7 Discussion Questions, we have used the modelS(t) =S(0)QN(t)i=1eXitodescribe the stock price, can you findE[S(t)/S(0)]? HereXiare i.i.d. exponential RVs with parameterμ;N(t)is a Poisson process with rateλ; andXiandN(t) are independent.Question 4:Trials are performed in sequence.Let us define a Markov Chain{Xn, n= 0,1,2, . . .}on the states given asbelow:Outcomes of the (n-1)th and nth trialsState ofXnSuccess, Success0Failure, Success1Failure, Failure2Success, Failure3If the last two trials were successes, the probability of being successful in the next trial is 0.8. Otherwise, theprobability of being successful in the next trial is 0.5. Then, the transition probability matrix can be obtainedasP=0BBB@0.8000.20.5000.500.50.5000.50.501CCCA(a) Do the limiting probabilities exist for this Markov Chain? Explain why.(b) In the long run, what is the proportion of trials being successful?1
Solution to Q4:a.It is irreducible as all states communicate. SinceP00= 0.8>0, state 0 is aperiodic. This implies that all statesare aperiodic as well. Also, in a finite MC, one of states must be a recurrent, so that all states are (positive)recurrent. Therefore, the limiting probabilities exist.b.SolvingπP=πandπ1= 1 whereπ= (π0, π1, π2, π3) andπi= limn→∞Pnjior equivalentlyπ0= 0.8π0+ 0.5π1π1= 0.5π2+ 0.5π3π2= 0.5π2+ 0.5π3∑3k=0πk= 1gives(π0, π1, π2, π3) =511,211,211,211.Then, the proportion of success islimn→∞Pr[(n+ 1)-th trial is successful] = limn→∞"3Xk=0Pr[(n+ 1)-th trial is successful|Xn=k] Pr(Xn=k)#=3Xk=0nlimn→∞Pr[(n+ 1)-th trial is successful|Xn=k] Pr(Xn=k)o=511×0.8 +211+211+211×0.5=711,where the second last equality is valid as the limits of all summands exist.2