Free Markov chain calculator: steady state from your transition matrix.
Name your states, set the transition probabilities one state at a time, and get the steady state distribution, the distribution after any number of steps, and a state diagram of the whole chain. Nothing leaves your browser.
1 Your states
A state is one situation the system can be in. Two to eight of them. The example below is the classic weather chain: a nice day, a rainy day, a snowy day.
2 Transition probabilities
3 The chain, drawn
4 Steady state
See how the steady state is reached
5 Where you are after n steps
See the calculation, step by step
The steady state uses the power method on your transition matrix. Your inputs stay in this browser.
From a transition matrix to the long run.
A Markov chain says the next state depends only on the current one. That one assumption is enough to answer a question no amount of staring at the matrix will: where does the system end up if it keeps running.
- List the states. Each state is one situation the system can be in, and the set has to cover every possibility.
- Fill in the transition matrix. Row i, column j is the chance of moving from state i to state j on the next step. Every row adds up to 1, because something always comes next.
- Read the steady state. The stationary distribution pi is the vector where pi times P equals pi. The calculator finds it by multiplying an even spread by the matrix over and over until it stops changing, which is the power method.
- Check a finite horizon. The n-step distribution answers a different question: starting from one known state, where is the chain likely to be after exactly n steps.
For the classic weather chain worked through by hand, step by step, see the Markov chain weather forecasting example in the documentation.
When one transition matrix is not the whole model.
This page solves a single fixed matrix. Real models carry money, time and decisions on top of the states, and that is what the desktop software is for.
- Rewards and costs attached to states and transitions
- Time-variant transition probabilities, not one fixed matrix
- Markov decision processes: choose the policy, not just observe the chain
- Cohort simulation over a horizon, with discounting
- Charts of the state occupancy over time
- Export the model and the results to a report
- Runs offline on Windows, no cloud dependency
Building a Markov model inside a larger sequential decision, for a health technology assessment or a staged investment, is a job for Decision Tree Software, which carries Markov models inside a decision tree.
Markov chains, transition matrices and the steady state.
What is a Markov chain?
A Markov chain is a model of a system that moves between a finite set of states, where the probability of the next state depends only on the current state and not on the path that led there. That property is called the Markov property, or memorylessness. Weather, machine condition, credit rating, customer status and disease progression are all commonly modeled this way.
What is a transition matrix?
The transition matrix P holds one row per state. The entry in row i, column j is the probability of moving to state j on the next step given that the chain is in state i now. Because something always happens next, every row has to add up to exactly 1. The diagonal entry is the probability of staying in the same state.
How is the steady state distribution calculated?
The steady state, or stationary distribution, is the row vector pi that satisfies pi times P equals pi with the components summing to 1. This calculator finds it by the power method: start from an even spread across the states, multiply by the transition matrix repeatedly, and the distribution converges to pi. For a chain where every state can eventually reach every other state and that is not periodic, the answer is unique and does not depend on where you start.
What does the n-step distribution mean?
If the chain starts in a known state, the n-step distribution is the probability of being in each state after n moves. It is computed by multiplying the starting distribution by the transition matrix n times. Early steps still remember where the chain started; as n grows the distribution drifts toward the steady state.
Why does a periodic Markov chain have no steady state?
A chain is periodic when returns to a state can only happen at multiples of some number greater than 1, for example a chain that strictly alternates between two states. The distribution then cycles forever instead of settling, so the power method never converges. The long-run share of time spent in each state is still well defined, and this calculator reports that instead, saying plainly that the chain is periodic.
Is this Markov chain calculator free?
Yes. It runs entirely in your browser, with no account, no sign-up, no usage limit and nothing sent to a server. For rewards and costs per state, time-variant transition probabilities and Markov decision processes, those live in the SpiceLogic Markov Chain Calculator desktop software.