What is a Markov chain


Next page:Recursive representation Upwards:Model description and examples Previous page:State space, initial distribution and transition probabilities & nbsp content


Examples

  1. Weather forecast
    (see O. Häggström (2002) Finite Markov Chains and Algorithmic Applications. CU Press, Cambridge)
    • We first consider regions in which longer rainy and dry periods typically alternate, with rainy days and sunny days occurring roughly equally on average.
      • Then it is relatively easy to predict the weather of the following day, if only the two "states" "Rain" resp. "Sunshine" can be considered.
      • If we assume that the forecast is correct 75% of all cases (regardless of whether it is raining or the sun is shining at the present day),
      • then the weather forecast can be modeled by a Markov chain with the following transition matrix:
        (7)

    • In regions in which this symmetry between "rain" and "sunshine" does not exist, but rather sunny days should occur much more often than rainy days
      • Weather forecast Not can be modeled by the transition matrix considered in (7).
      • In this case, for example, the transition matrix
        (8)

        be a suitable model.
  2. Random wanderings; Risk processes 
    • Classic examples of Markov chains are through so-called random wanderings given that also in German-language literature Random Walk where the (unrestricted) basic model is given in the following way.
    • Notice
  3. Queues
    • The number of customers waiting in front of any given, but fixed, checkout in a supermarket can be modeled as follows using a Markov chain.
    • The recursively defined sequence of random variables with
      (10)

      then forms a Markov chain, its transition matrix is given by
    • It is the random number of customers in the queue, immediately after this the operation of the -th customer was terminated (i.e. without the consideration of the customer whose service may have just started and who has therefore already left the queue).
  4. Branching processes
  5. Cyclical random wanderings 
    • Further examples of Markov chains can be constructed as follows (see E. Behrends (2000) Introduction to Markov Chains. Vieweg, Braunschweig, p.4).
      • We consider the finite state space , the initial distribution
        (12)

        and the transition probabilities
      • Be independent random variables, where the distribution of given by (12) and
      • The recursively defined sequence of random variables with
        (13)

        For then forms a Markov chain that cyclic random walk is called; see also exercise 1.3.
    • Notice
      • This model of a Markov chain can be done in the following way experimental be realized: we first throw -times a coin and register how often the event "number" occurs. The number We understand these events as the realization of the random initial state on; see that Bernoulli scheme in section WR-3.2.1.
      • After that we throw -times a dice and register the respective numbers. The number that at -th roll of the die occurs, we consider the realization of the random "growth" on; .
      • The new "system state" then results from the "update" of the old system status according to (13), taking into account the "increase" .
      • If the experiment is not actually carried out by throwing a coin or a dice, but by generating the appropriate Pseudo random numbers is carried out with the computer, then one speaks of Monte Carlo simulation.
      • Methods for constructing dynamic simulation algorithmswhich are based on Markov chains are dealt with in detail in the second part of the lecture.


Next page:Recursive representation Upwards:Model description and examples Previous page:State space, initial distribution and transition probabilities & nbsp content Ursa Pantle 2003-09-29