QALY and Cost with Markov Model

Let's define a situation of a cohort of 100,000 people. There are four states this cohort goes thru. "Well", "Recurrence", "Dead" and "Dying by disease". We are interested to find out the expected cost and expected QALY of this cohort. The transition probability of these four states is defined as

Four-state transition diagram for the QALY and Cost example: Well, Recurrence, Dying by disease, and Dead, with all between-state transition probabilities labelled.

The Life year gained for state Well and Recurrence is 1. The cost of staying in Well is 350$. The cost for staying in recurrence is 3190$. The cost of dying by disease is 1400$. There is no cost associated with the dead state.

Let's find out the life expectancy of this cohort. Also, let's find out the expected cost of this cohort. Start the Rational Will or SpiceLogic Decision Tree software and click the Markov Model button.

Decision Tree Analyzer start page used to launch the Markov Model wizard for the QALY and Cost cohort example.

Step 1: Specifying States

Once you clicked the Markov Model button, you will be presented with the Wizard, where you can add 4 states as shown below.

Markov wizard step 1 with all four states (Well, Recurrence, Dying by disease, Dead) added for the QALY and Cost analysis.

Then, click "Proceed". On the next screen, you will be asked if there will be any action from a state. Simply answer "No" and continue to Proceed.

Wizard prompt 'Your control in a State' asking whether the user's actions can influence the probability of transitioning from the Well state, with Yes / No / Back / Cancel buttons.

Step 2: Configuring Cohort Simulation Settings

Then, you will be asked to set the Markov Decision Process Simulation settings. Set the settings as shown below. Set the maximum number of state iterations to 55.

Markov cohort simulation settings in the wizard with the maximum number of state iterations set to 55 for the QALY and Cost example.

Step 3: Setting up transition probabilities

Once you click Proceed you will be asked to set the transition probabilities. Set the transition probabilities for all "Well" states as shown below. Then Proceed.

Transition probability editor for the Well state with probabilities entered for transitions to Recurrence, Dying by disease, and Dead.

In the next screen, you will be asked to set the transition probability for another state, which can be "Recurrence". Set it as shown below.

Transition probability editor for the Recurrence state, configuring the probabilities of moving to the other three states in the next cycle.

Then click Proceed. This time, you may be asked to set the transition probabilities for the state "Dying by disease". Set the transition probabilities for this state. Then click proceed.

Transition probability editor for the Dying by disease state, defining the probability of progressing to Dead versus remaining in the same state.

Finally, mark the "Dead" state as an absorbing state. Then click "Proceed".

Dead state marked as an absorbing state so once the cohort enters Dead it cannot transition out, closing out the simulation for that fraction.

Step 4: Specifying the initial state

From the previous step, you will be taken to the page where you can specify the initial state. Select the state "Well" as the initial state.

Initial-state step of the wizard with the Well state selected as the certain starting state for the cohort.

Step 5: Setting Payoff

Now is the time to set the Payoff for the states. Click "Yes" on the following screen.

Wizard prompt for adding payoffs to the states, answered Yes so cost and life-year payoffs can be configured next.

Then in the next step, click the button "Cost-Effectiveness Analysis in Healthcare".

Criteria type selection step with the 'Cost-Effectiveness Analysis in Healthcare' button highlighted as the payoff framework for this Markov model.

Then, set the Effectiveness criterion as shown below.

Effectiveness criterion configuration with Life-years selected as the effectiveness measure used to compute expected life-years for the cohort.

Now select the "Cost" tab header and check the box as shown below.

Cost Criterion tab with the Minimize Cost checkbox enabled, Unit $, Minimum Possible Cost 0, Maximum Possible Cost 10000, and Maximum willing to pay per unit of effectiveness 50000.

Click Proceed. You will be asked to set the Cost and Effectiveness for the "Well" state. Set Life-years = 1, as every time the "Well" state is visited, 1 life year is added. Set the cost as 350$.

Payoff editor for the Well state with Life-years set to 1 and cost set to 350 dollars per cycle the cohort stays in Well.

Then click Proceed. Set the state cost-effectiveness of Recurrence as 1 life year, and 3190$ cost. Click proceed. For the state Dying by disease, set life year 0 and cost 1400$. Then click "Proceed". Finally for the Dead state, no cost and no effectiveness needed to be set. So, simply click the "Skip setting reward for this state" button.

Reward step for the Dead state with the 'Skip getting a reward for this state' button used, since no cost or effectiveness accrues in Dead.

Once you click that button, you will see your final model is created as a decision tree diagram.

Step 6: Analyzing the result

Now, open the Markov Analyzer panel.

Completed QALY and Cost Markov model shown as a decision tree diagram with the Markov Analyzer panel ready to inspect cohort results.

The Life Expectancy and Expected cost of this cohort can be viewed as a tooltip of the Markov Chance Node as shown below.

Tooltip on the Markov Chance node displaying the expected life-years and expected cost computed for the cohort across the simulation.

And you can view all the charts from various perspectives of this Markov model.

Markov Analyzer charts window for the QALY and Cost model with probability traces, cumulative cost, and effectiveness charts across cycles.

Step 7: Modifying / Refining the model

Once you have completed the wizard step-by-step user interface for creating your Markov model, you will see the decision tree showing the Markov Process Diagram. Now, you can change the States, Transition Probabilities, Rewards, anything or everything. Please learn how to work on the diagram for modification from this page.