QALY and Cost with Markov Model

Let's define a situation of a cohort of 100,000 people. There are four states this cohort goes thru. "Well", "Recurrence", "Dead" and "Dying by disease". We are interested to find out the expected cost and expected QALY of this cohort. The transition probability of these four states is defined as 

The Life year gained for state Well and Recurrence is 1. The cost of staying in Well is 350$. The cost for staying in recurrence is 3190$. The cost of dying by disease is 1400$. There is no cost associated with the dead state.

Let's find out the life expectancy of this cohort. Also, let's find out the expected cost of this cohort. 

Start with the Markov Decision Process software. You will see a wizard shows up like this: 

Step 1: Specifying States

Add 4 states as shown below.

Then, click "Proceed". On the next screen, you will be asked if there will be any action from a state. Simply answer "No" and continue to Proceed. 

Step 2: Configuring Cohort Simulation Settings

Then, you will be asked to set the Markov Decision Process Simulation settings. Set the settings as shown below. Set the maximum number of state iterations to 55. 

Step 3: Setting up transition probabilities

Once you click Proceed you will be asked to set the transition probabilities. Set the transition probabilities for all "Well" states as shown below. Then Proceed. 

In the next screen, you will be asked to set the transition probability for another state, which can be "Recurrence". Set it as shown below. 

Then click Proceed. This time, you may be asked to set the transition probabilities for the state "Dying by disease". Set the transition probabilities for this state. Then click proceed. 

Finally, mark the "Dead" state as an absorbing state. Then click "Proceed". 

Step 4: Specifying the initial state

From the previous step, you will be taken to the page where you can specify the initial state. Select the state "Well" as the initial state.

Step 5: Setting Payoff

Now is the time to set the Payoff for the states. Click "Yes" on the following screen.

Then in the next step, click the button "Cost-Effectiveness Analysis in Healthcare". 

Then, set the Effectiveness criterion as shown below. 

Now select the "Cost" tab header and check the box as shown below. 

Click Proceed. You will be asked to set the Cost and Effectiveness for the "Well" state. Set Life-years = 1, as every time the "Well" state is visited, 1 life year is added. Set the cost as 350$. 

Then click Proceed. Set the state cost-effectiveness of Recurrence as 1 life year, and 3190$ cost. Click proceed. For the state Dying by disease, set life year 0 and cost 1400$. Then click "Proceed". Finally for the Dead state, no cost and no effectiveness needed to be set. So, simply click the "Skip setting reward for this state" button. 

Once you click that button, you will see your final model is created as a decision tree diagram. 

Step 6: Analyzing the result

Now, open the Markov Analyzer panel.

The Life Expectancy and Expected cost of this cohort can be viewed as a tooltip of the Markov Chance Node as shown below.

And you can view all the charts from various perspectives of this Markov model. 

Step 7: Modifying / Refining the model

Once you have completed the wizard step-by-step user interface for creating your Markov model, you will see the decision tree showing the Markov Process Diagram. Now, you can change the States, Transition Probabilities, Rewards, anything or everything. Please learn how to work on the diagram for modification from this page.