The free Markov chain calculator.
Build a transition matrix and instantly see its steady state, any n-step distribution, and a state diagram.
1 Your states
A state is one situation the system can be in. Two to eight of them. The example below is the classic weather chain: a nice day, a rainy day, a snowy day.
2 Transition probabilities
The chain, drawn
Steady state
See how the steady state is reached
How the probabilities change, step by step
The steady state uses the power method on your transition matrix. Your inputs stay in this browser.
From a transition matrix to the long run.
A Markov chain says the next state depends only on the current one. That one assumption is enough to answer a question no amount of staring at the matrix will: where does the system end up if it keeps running.
- List the states. Each state is one situation the system can be in, and the set has to cover every possibility.
- Fill in the transition matrix. Row i, column j is the chance of moving from state i to state j on the next step. Every row adds up to 1, because something always comes next.
- Read the steady state. The stationary distribution pi is the vector where pi times P equals pi. The calculator finds it by multiplying an even spread by the matrix over and over until it stops changing, which is the power method.
- Check a finite horizon. The n-step distribution answers a different question: starting from one known state, where is the chain likely to be after exactly n steps.
For the classic weather chain worked through by hand, step by step, see the Markov chain weather forecasting example in the documentation.
Go beyond a simple Markov chain.
This page answers one question: given a fixed transition matrix, where does the chain go. A real decision carries money, time and alternatives on top of the states, and that takes a decision model, not a bigger calculator. Decision Tree Pro builds the complete decision analysis around your Markov process.
- Attach rewards and costs to Markov states
- Turn a chain into a Markov decision process
- Time-variant transition probabilities, as a function or a look-up table
- Cohort simulation over a horizon, with charts and tables
- Markov models inside a decision tree, beside your alternatives
- Sensitivity analysis and Monte Carlo simulation
- Cost-effectiveness analysis with QALY or DALY payoffs
Decision Tree Pro is also part of SpiceLogic Rational Will, the suite that adds AHP, Bayesian inference and multi-criteria analysis.
Prefer a Windows desktop app? The free SpiceLogic Markov Chain Calculator models the same simple, time-invariant chains offline, with a guided wizard, charts and Excel export.
Get the free desktop calculatorMarkov chains, transition matrices and the steady state.
What is a Markov chain?
A Markov chain is a model of a system that moves between a finite set of states, where the probability of the next state depends only on the current state and not on the path that led there. That property is called the Markov property, or memorylessness. Weather, machine condition, credit rating, customer status and disease progression are all commonly modeled this way.
What is a transition matrix?
The transition matrix P holds one row per state. The entry in row i, column j is the probability of moving to state j on the next step given that the chain is in state i now. Because something always happens next, every row has to add up to exactly 1. The diagonal entry is the probability of staying in the same state.
How is the steady state distribution calculated?
The steady state, or stationary distribution, is the row vector pi that satisfies pi times P equals pi with the components summing to 1. This calculator finds it by the power method: start from an even spread across the states, multiply by the transition matrix repeatedly, and the distribution converges to pi. For a chain where every state can eventually reach every other state and that is not periodic, the answer is unique and does not depend on where you start.
What does the n-step distribution mean?
If the chain starts in a known state, the n-step distribution is the probability of being in each state after n moves. It is computed by multiplying the starting distribution by the transition matrix n times. Early steps still remember where the chain started; as n grows the distribution drifts toward the steady state.
Why does a periodic Markov chain have no steady state?
A chain is periodic when returns to a state can only happen at multiples of some number greater than 1, for example a chain that strictly alternates between two states. The distribution then cycles forever instead of settling, so the power method never converges. The long-run share of time spent in each state is still well defined, and this calculator reports that instead, saying plainly that the chain is periodic.
Is this Markov chain calculator free?
Yes. It runs entirely in your browser, with no account, no sign-up, no usage limit and nothing sent to a server. If you would rather work offline, the free SpiceLogic Markov Chain Calculator for Windows models the same simple, time-invariant chains as an installed desktop app. Rewards and costs on states, time-variant transition probabilities and Markov decision processes belong to Decision Tree Pro.