# What is a Markov chain

**Next page:**Recursive representation

**Upwards:**Model description and examples

**Previous page:**State space, initial distribution and transition probabilities & nbsp

**content**

Examples

*Weather forecast*

(see O. Häggström (2002)*Finite Markov Chains and Algorithmic Applications*. CU Press, Cambridge)- We first consider regions in which longer rainy and dry periods typically alternate, with rainy days and sunny days occurring roughly equally on average.
- Then it is relatively easy to predict the weather of the following day, if only the two "states" "Rain" resp. "Sunshine" can be considered.
- If we assume that the forecast is correct 75% of all cases (regardless of whether it is raining or the sun is shining at the present day),
- then the weather forecast can be modeled by a Markov chain with the following transition matrix:
(7)

- In regions in which this symmetry between "rain" and "sunshine" does not exist, but rather sunny days should occur much more often than rainy days
- Weather forecast
*Not*can be modeled by the transition matrix considered in (7). - In this case, for example, the transition matrix
(8)

be a suitable model.

- Weather forecast

- We first consider regions in which longer rainy and dry periods typically alternate, with rainy days and sunny days occurring roughly equally on average.
*Random wanderings; Risk processes*- Classic examples of Markov chains are through so-called
*random wanderings*given that also in German-language literature*Random Walk*where the (unrestricted) basic model is given in the following way. - Notice

- Classic examples of Markov chains are through so-called
*Queues*- The number of customers waiting in front of any given, but fixed, checkout in a supermarket can be modeled as follows using a Markov chain.
- The recursively defined sequence of random variables with
(10)

then forms a Markov chain, its transition matrix is given by - It is the random number of customers in the queue, immediately
*after this*the operation of the -th customer was terminated (i.e.*without*the consideration of the customer whose service may have just started and who has therefore already left the queue).

*Branching processes**Cyclical random wanderings*- Further examples of Markov chains can be constructed as follows (see E. Behrends (2000)
*Introduction to Markov Chains*. Vieweg, Braunschweig, p.4).- We consider the finite state space , the initial distribution
(12)

and the transition probabilities - Be independent random variables, where the distribution of given by (12) and
- The recursively defined sequence of random variables with
(13)

For then forms a Markov chain that*cyclic random walk*is called; see also exercise 1.3.

- We consider the finite state space , the initial distribution
- Notice
- This model of a Markov chain can be done in the following way
*experimental*be realized: we first throw -times a coin and register how often the event "number" occurs. The number We understand these events as the realization of the random initial state on; see that*Bernoulli scheme*in section WR-3.2.1. - After that we throw -times a dice and register the respective numbers. The number that at -th roll of the die occurs, we consider the realization of the random "growth" on; .
- The new "system state" then results from the "update" of the old system status according to (13), taking into account the "increase" .
- If the experiment is not actually carried out by throwing a coin or a dice, but by generating the appropriate
*Pseudo random numbers*is carried out with the computer, then one speaks of*Monte Carlo simulation*. - Methods for constructing
*dynamic simulation algorithms*which are based on Markov chains are dealt with in detail in the second part of the lecture.

- This model of a Markov chain can be done in the following way

- Further examples of Markov chains can be constructed as follows (see E. Behrends (2000)

**Next page:**Recursive representation

**Upwards:**Model description and examples

**Previous page:**State space, initial distribution and transition probabilities & nbsp

**content**Ursa Pantle 2003-09-29

- What is a failure in deep learning
- First class tickets are refunded
- What are Marketing Segmentation Strategies
- How bad is second hand smoke
- What are the side effects of midazolam
- Which NASA missions failed?
- What are the limitations of Google Translate
- Which hospital systems are profitable
- What types of software projects are there
- Why was Burma renamed Myanmar
- What is your opinion on benzodiazepines
- Is cafeteria eating healthy
- Would a hike in the minimum wage make a difference?
- What is m2m
- How we can study physics
- Why are robotic vacuums round
- Some people try to be overweight
- Why does the gun barrel have tiny holes?
- How do you solve business problems effectively
- What are the most unhealthy pasta dishes
- Does sex influence love
- Finasteride Propecia can cause prostate cancer
- How are people tied to time
- Does the interest rate extension work in Facebook ads