(aconstruct computer program to simulate the markov chainand show realization for steps (bcompute the limiting probabilities that the markov chain is in state , , by solving the global balance equations ( (cverify that the exact limiting probabilities correspond to the average fraction of times that the markov process visits states , , for large number of steps as generalization of example consider random walk on an arbitrary undirected connected graph with finite vertex set for any vertex vlet (vbe the number of neighbors of -called the degree of the random walk can jump to each one of the neighbors with probability / (vand can be described by markov chain show thatif the chain is aperiodicthe limiting probability that the chain is in state is equal to ( ) ( let uv ~iid ( the reason why in example the sample mean and sample median behave very differently is that [ /vwhile the median of / is finite show thisand compute the median [hintstart by determining the cdf of / by writing it as an expectation of an indicator function consider the problem of generating samples from gamma( (adirect simulationlet ~iid ( show that ln( )/ -ln( )/ gamma( [hintderive the distribution of ln( )/ and use example (bsimulation via mcmcimplement an independence sampler to simulate from the gamma( target pdf ( - using proposal transition density ( xg( )where (yis the pdf of an exp( random variable generate samplesand compare the true cdf with the empirical cdf of the data let [xy]be random column vector with bivariate normal distribution with expectation vector [ ]and covariance matrix sa (awhat are the conditional distributions of ( xand ( )[hintuse theorem (bimplement gibbs sampler to draw samples from the bivariate distribution (usfor and and plot the resulting samples here the objective is to sample from the -dimensional pdf (xyc -(xy+ +yx for some normalization constant cusing gibbs sampler let (xyf