Calculating Life Expectancy with time variant probabilities

In this example, we will show how to utilize the Markov Chain to calculate the life expectancy of a cohort.

Say, a cohort of 10,000 people has a probability of death from year 0 years to 100 years as shown in the following table.

Sample data table for the life expectancy example, mapping age in years (0 to 100) to the probability of death used as the time-variant Markov transition.

Let's start with the Rational Will or Decision Tree Software, and choose "Markov Model" from this start screen.

Decision Tree Analyzer start screen with the Markov Model button highlighted, used to begin the life expectancy Markov chain example.

Please note that we are choosing a Markov node as the root node of the Decision Tree because, in this example, we will focus on the Markov Chain only, and not any other decision analysis. But, once the model is created, you can add other decision nodes later for performing more analysis. Anyway, you will see a wizard shows up like this:

Opening screen of the Markov Model wizard launched after choosing a Markov node as the root of the tree for the life expectancy example.

Step 1: Creating the Markov States

In this wizard, add two states named "Well" and "Dead". Click Proceed.

Markov wizard step 1 with two states named Well and Dead added to the chain for the life expectancy cohort model.

Then, you will be presented with the following question.

Wizard prompt 'Your control in a State' asking whether the user's actions can influence the probability of transitioning from the Well state to another state, with Yes / No / Back / Cancel options.

In simple words, it is asking, do you want to use Action in your Markov Model. For this project, we do not have Markov Action, rather we have only Markov states. So, click No. Then you will be asked the same question for the state "Dead". Answer No as well.

Step 2: Setting up Markov Simulation Process

Once you clicked 'No' in the previous step, you will be asked to set up your Markov simulation setting. Markov Model is solved by Cohort Simulation, and you can configure the simulation parameter. When you are done specifying your Markov States and Actions, you will be presented with the following screen to configure your Markov simulation. For our simulation, let's use Half Cycle Correction, so check the box for "Half Cycle Correction". For our example, we need to set the number of iterations to 21, and the duration for each cycle = 5 years. You can set the Cohort Size to 10,000 even though the result won't vary much based on the various cohort size. The higher the number is, the more accurate the result will be. For this case, 10,000 is good enough.

Cohort simulation settings step in the wizard with cycle count, cycle name, and cohort size configured for the life expectancy run.

Now, click "Proceed".

Step 3: Setting transition probabilities (importing from Excel)

After clicking proceed from the previous step, you will be asked to set up your transition probabilities.

Transition probability step with the 'At every cycle, the probability depends on the cycle number (time-variant)' checkbox available for age-driven probabilities.

For our case, we have a variable transition probability that depends on age. So, check the box "At every cycle, the probability depends on the cycle number (time-variant)" as shown in the above screenshot. Once you check that box, you will be presented with the following screen. This is a carousel of all state transitions.

Carousel view of time-variant transition probabilities, letting the user step through each from-to state pair to configure its cycle-dependent probability.

From the carousel, select the transition for Well-Dead. If you see the "Automatically calculate as the complement of other event's probability" box checked for the "Well - Dead" transition, uncheck that box.

Well to Dead transition view with the 'Automatically calculate as complement' checkbox cleared so the probability can be specified directly from the data.

Then, click the "Look up table" button. Then click the button for "Lookup table". You will see a view like this. In this view, notice that there is

Look-up table view for the Well to Dead transition, with columns for cycle number (age) and the corresponding probability value.

Then, open your Excel file and select the columns where the first column contains the Cycle number and the second column contains the probability. After selection, Copy the data to your clipboard.

Excel worksheet with the age (cycle number) column and corresponding probability column selected and copied to the clipboard for import.

Now, get back to the Decision Tree software and click this button to import the table from your clipboard. Please make sure that your data format is correct as shown in the above screenshot.

Import-from-clipboard button on the look-up table editor used to paste the age and probability columns previously copied from Excel.

Once you click that import button, you will see the data is imported as shown below.

Look-up table populated with age-to-death probabilities imported from the Excel clipboard, ready to drive the time-variant Markov transition.

Now, Click "Proceed". You will be asked to set the transition probabilities for the "Dead" state. As you know, "Dead" is an absorbing state, which means, the probability of transitioning to any other state from the "Dead" state is 0. So, check this box that says "Dead is an absorbing state".

Transition setup for the Dead state with the 'Dead is an absorbing state' checkbox enabled so no transition can leave Dead in the simulation.

After checking that box, the transition probability controls will be hidden. Now, click Proceed.

Transition controls hidden after the Dead state is marked absorbing, confirming the wizard accepted the absorbing-state configuration.

Step 4: Setting Initial State

Once you click "Proceed" from the previous state, you will be asked to set your initial state. You can either set a certain initial state or a probability of the initial state.

Wizard step for choosing either a certain initial state or a probability distribution across states for where the cohort starts.

In the following screenshot, you can see what it will look like if you choose Uncertain Initial state.

Uncertain initial state view showing per-state probability inputs that sum to 1 when the cohort does not start in a single known state.

But for our model, keep it back to a Certain Initial State for the "Well" state.

Step 5: Setting Payoff or Reward

Once you click the Proceed button, you will be asked if you want to set Payoff for your state. In our case, yes, we want to set a payoff. The payoff is the life year. In every iteration, we land to the state "Well", we gain 5 life years. So, click "Yes" on the following screen.

Wizard prompt asking whether to attach a payoff to the Markov states, answered Yes so a life-year reward can be added to the Well state.

Then, choose the "Cost-Effectiveness Analysis in Healthcare" button.

Objective type selection step in the wizard with the 'Cost-Effectiveness Analysis in Healthcare' button highlighted as the payoff framework.

Then, the cost-effectiveness setup window will appear.

Cost-effectiveness setup window with the custom variable radio option selected to define a domain-specific effectiveness measure.

Once you check that radial box for a custom variable, you can define your variable as shown below.

Custom effectiveness variable defined as Life Year for the life expectancy Markov model, including name, symbol, and unit fields.

For this analysis, we do not need to configure Cost. So, just click Proceed after that. Then, you will be asked to set the reward in the "Well" State. Set the Life year "5" for the state "Well".

Reward editor for the Well state with the Life Year value set to 5, awarded each cycle the cohort remains in Well.

Then, click "Proceed". You will be asked to set a reward for the "Dead" state. You can either set the value to 0 or click the button "Skip getting a reward for this state". Once you do that, you will see your final Markov Model ready as a Decision Tree Diagram.

Step 6: Analyzing Result

In the diagram, notice that the expected value is displayed over the node. We can see the expected life year is shown as 76.87. So, 76.87 is the life expectancy of this cohort.

Completed life expectancy Markov model rendered as a decision tree diagram, with the simulation's expected value displayed at the root.

Once you click on the pop-out button, you will see the charts shown as a grid.

Pop-out window displaying every Markov analyzer chart for the life expectancy run laid out as a single scrollable grid.

If you want to see the data table behind any chart, you can right-click on that chart and see the data table, which you can also export in Excel.

Data table behind a Markov life expectancy chart, exposed via right-click and exportable to Excel for further analysis.

But, you can get the organized Cohort simulation traces from this panel.

Cohort simulation trace table listing, for each cycle, the population in each state across the lifetime of the life expectancy Markov run.

Step 7: Modifying / Refining the model

Once you have completed the wizard step-by-step user interface for creating your Markov model, you will see the decision tree showing the Markov Process Diagram. Now, you can change the States, Transition Probabilities, Rewards, anything or everything. Please learn how to work on the diagram for modification from this page.