How To - Markov Models

Today in preparation for my upcoming exam I want to show you how to work with Markov Models.
This is one of the easiest parts of statistics, so bear with me and we get through this in no less then 5min ;)

Ok and I haven't yet figured out the HTML code to align my pictures correctly - so for all OCDs out there, I am so terribly sorry...

In a Markov Model, we work with states and their respective probability to transfer into a different state.
In our example you see the states written as z (z1, z2, z3) and their probabilities written in green (i.e. state z1 has a transition probability of 1 to transfer into state z2).
You can also see, that states can choose to stay in themselves, and not transfer into another state - with an arrow circling back to them.

We will create a matrix named M to store our transition values, this makes it easier to calculate with them.

Note: because we are working with probabilites, the values we work with can range 
from 0 to 1 (it doesn't happen or it happens 100%).
That's also why each column must add up to a total of 1 - also that's an easy way to check if you screwed up writing down the values during an exam...

Imagine now, that we are playing a computer game and we only got about 10min left before we REALLY got to get our beauty sleep and we want to know if we might make it through the next 3 levels (level 1 = z1, level2 = z2, level 3 = z3).

To calculate our chances, for each level we use the Falk Schema:
Vektor V represents the level we wish to beat, and Vektor V1 our estimated chances to make it...
Please keep in mind that these are your chances if you play the game on the first round.
If you want to keep trying, you will have to multiply Matrice M with Vektor V1 to get your chances for the second round, and so on...

I'm gonna try my birthday game Total War Rome II now^^
Anybody care to join me?


Popular posts from this blog

How To - CYK Algorithm

How To - Citric Acid Cycle

How To -Glycolysis - Biochemistry