Let me tell you about a purchasing manager I will call Dana, because the way her week went is the fastest way I know to explain when AHP earns its keep and when it does not.
Dana walked into her office on a Tuesday morning to six supplier bid folders stacked on her desk and a sticky note from her finance director on top of them. It said: cheapest wins unless you can show me why not. Friday.
She already knew which folder was cheapest. Supplier E, by a clear margin. She also had a strong feeling that E was the wrong choice, and she knew exactly how far a feeling gets you with a finance director. Nowhere.
What Dana actually had to decide
Six candidate suppliers, which I will call A through F so the arithmetic stays readable. Five things that mattered for this contract:
- Unit price, because someone has to sign the invoice.
- Quality, measured the way her plant already measured it: incoming inspection pass rate.
- Lead time, from purchase order to dock.
- Financial stability, because a supplier that goes under mid-contract costs more than any unit price saving ever recovered.
- Support responsiveness, the one everybody forgets until the line stops at two in the morning.
Look at that list for a second. Those five are not equally important, and they are not measured in the same units. You cannot add dollars to a pass rate. That is the entire reason Dana could not just build a spreadsheet and total the columns, and it is the entire reason a weighting method exists.
Tuesday afternoon: eighty-five questions
Dana remembered the Analytic Hierarchy Process from an MBA module. Compare things in pairs, get weights out, the software does the matrix algebra. She set up the five criteria and six suppliers and started answering pairwise questions. Is quality more important than price, and by how much? Is supplier A better than supplier B on lead time, and by how much?
Here is the arithmetic she had not done in advance. Comparing n items in pairs takes n(n-1)/2 judgments. Six suppliers is 15 comparisons, and you make those 15 for every criterion. Five criteria means 75 comparisons on the suppliers alone. Add the 10 needed to weigh the five criteria against each other and you are at 85 separate judgments.
Somewhere around comparison number thirty, Dana noticed she had stopped reading the two supplier names and started clicking whichever answer got her to the next screen. So she stopped.
That is the part of this story worth remembering. A rushed pairwise matrix does not fail loudly. It does not throw an error. It produces a ranking that looks exactly as authoritative as a careful one, and nobody in the Friday meeting can tell the difference.
Wednesday: rate them, do not compare them
The software had a second mode she had skipped past. Instead of comparing pairs, it asked her to put each thing on a 0 to 100 scale directly. How important is quality? Ninety. How good is supplier C on lead time? Eighty-five. The method is absolute measurement in the SMART tradition (Edwards and Barron, 1994), and our AHP Software ships it as Direct Rating AHP.
The arithmetic underneath is deliberately simple. Each criterion's weight is its rating divided by the sum of all the ratings:
wi = ratingi / Σ ratingj
Each supplier's priority on a criterion is its rating divided by the sum of all six suppliers' ratings on that criterion:
rij = ratingij / Σk ratingkj
And the final score is the weighted sum, Σi wi × rij.
Count the inputs. Five criterion ratings, plus six suppliers on five criteria, so thirty supplier ratings. Thirty-five numbers instead of eighty-five judgments. Dana had the whole model entered before lunch.
Step 1: how much does each criterion matter?
She did not do this alone. She pulled the plant manager into her office and they argued for twenty minutes about whether quality or price should be the 100. Quality won, narrowly, on the grounds that a bad part costs more to find than a cheap part saves. These are the numbers they settled on:
| Criterion | Importance (0 to 100) | Weight |
|---|---|---|
| Quality | 100 | 0.286 |
| Unit price | 90 | 0.257 |
| Lead time | 70 | 0.200 |
| Financial stability | 50 | 0.143 |
| Support responsiveness | 40 | 0.114 |
| Total | 350 | 1.000 |
Divide each rating by 350 and you have weights that add to 1. Quality at 100 and price at 90 means quality carries about 11 percent more of the final answer. That twenty-minute argument is now a number, and the number is in the appendix where the director can see it.
Step 2: score every supplier on every criterion
Same 0 to 100 scale, one criterion at a time. Higher always means better. That sounds obvious until you get to price.
The first time through, Dana scored supplier E's price as 40, because the number on E's quote was low. Then she looked at the column and realised she had it backwards. A low price is a good price, so E's rating had to be high. She fixed it. The model would not have caught it for her, and if she had left it, the cheapest supplier would have been punished for being cheap. This is the single most common mistake I see in direct rating, and it is worth a slow second look at every column before you move on.
| Supplier | Price | Quality | Lead time | Stability | Support |
|---|---|---|---|---|---|
| A | 60 | 95 | 70 | 90 | 80 |
| B | 85 | 70 | 60 | 55 | 75 |
| C | 40 | 90 | 85 | 80 | 60 |
| D | 75 | 65 | 90 | 60 | 85 |
| E | 90 | 50 | 55 | 45 | 65 |
| F | 55 | 80 | 65 | 70 | 50 |
| Column total | 405 | 450 | 425 | 400 | 415 |
Each column is normalized by its own total. Supplier A on price is 60/405 = 0.148. On quality, 95/450 = 0.211. Every column now sums to 1, which is what makes a price rating and a quality rating comparable at all.
Step 3: multiply and add
Multiply each normalized rating by its criterion weight and add the five results. Here is supplier A in full, because I want you to see there is nothing hidden in the machinery:
- Price: 0.257 × 0.148 = 0.038
- Quality: 0.286 × 0.211 = 0.060
- Lead time: 0.200 × 0.165 = 0.033
- Financial stability: 0.143 × 0.225 = 0.032
- Support: 0.114 × 0.193 = 0.022
Supplier A totals 0.186. The same five lines for the other five suppliers gives the ranking:
| Rank | Supplier | Score |
|---|---|---|
| 1 | A | 0.186 |
| 2 | D | 0.176 |
| 3 | C | 0.168 |
| 4 | B | 0.167 |
| 5 | F | 0.155 |
| 6 | E | 0.149 |
The six scores sum to 1.000, so you can read them as shares of the total value on the table. Supplier A carries 18.6 percent of it.
Thursday: the cheapest supplier finished last
Dana looked at that table for a long time. Supplier E, the one on the sticky note, had the best price of the six by a wide margin, rated 90 where the winner only managed 60. And E finished sixth.
She built one picture for Friday. Rank by price on the left, rank by final score on the right, and a line for each supplier connecting the two.
The model also told her exactly why, and this is the sentence she wrote on an index card. Price carries 0.257 of the decision. Quality and financial stability together carry 0.429. E scored 50 and 45 on those two. The price advantage was real, and it was not big enough to cover the gap on everything else.
That sentence is the entire value of the exercise. Not the ranking, which Dana could have guessed on Tuesday. The ability to say why, with the arithmetic attached, to somebody who walks in sceptical.
Friday, 9am: the question she did not have an answer for
The director looked at the chart for a while, nodded, and asked exactly one question. How much would A's quality have to drop before E wins?
Dana did not know. She had a ranking. She did not have its breaking point.
She went back to her desk and ran the sensitivity analysis, which is the part of AHP most people skip and the part that actually closes a decision. It moves each input and reports how far it has to travel before the winner changes. For this model, the answer was that supplier A's quality rating would have to fall from 95 to below 70 before E overtook it, and nothing in E's own bid moved the ranking at all. She sent that as a two-line email at 10:40. The contract went to A.
The ranking got her into the room. The sensitivity number is what let her leave it.
One more thing she noticed while she was in there. Suppliers C and B scored 0.168 and 0.167. Do not read that as "C beats B". A one thousandth gap is inside the noise of anybody's 0 to 100 judgment, and rating one criterion five points differently flips it. Treat close scores as a tie and break them on something the model does not contain: capacity next quarter, distance to the plant, who you already have a relationship with. A method that pretends to resolve a 0.001 difference is lying to you.
What Dana gave up, and knew it
Direct rating is faster, and speed is not free. Here is the honest accounting, because Dana's director eventually asked about this too.
Standard pairwise AHP is redundant on purpose. Because you compare A to B, B to C, and also A to C, the model can check whether your judgments hang together and report a consistency ratio. If you say A is twice B, B is three times C, and then rate A as roughly equal to C, it catches you. That is a real safety net, and it has changed decisions that were about to be made on a muddled set of preferences.
Direct rating has no such redundancy. Each number is entered once and stands alone, so there is nothing to cross-check. Our implementation reports consistency as not applicable in this mode rather than printing a consistency ratio that would mean nothing. A fabricated 0.000 would look reassuring and tell you nothing about the quality of the inputs.
You also lose the elicitation benefit. People are measurably better at saying "this is somewhat more important than that" than at saying "this is a 70". Pairwise comparison exists because it asks an easier question. Dana traded that for speed, and for six suppliers on a Friday deadline, that was the right trade.
The rule Dana uses now
She has run three more of these since. Her rule of thumb:
- Up to four or five options, and the decision is expensive or contested: standard pairwise AHP. The comparison count is still tolerable and the consistency ratio is worth having.
- Six options or more, or a long list to screen: Direct Rating AHP first, then standard pairwise on the two or three survivors.
- Experienced raters who are short on time: direct rating, because a rushed pairwise matrix is worse than a careful rating.
Screen wide and cheap. Compare carefully among the finalists. You get the scale of rating and the rigour of pairwise exactly where each one pays.
Doing this yourself
Everything Dana did is arithmetic you can reproduce in a spreadsheet, and for one decision with five criteria you probably should. Software earns its place when the model stops being small: sub-criteria under criteria, several people rating independently and needing their judgments combined, or the question the director always asks next, which is how far something has to move before the answer changes.
Our AHP Software carries both modes, so you can screen with Direct Rating AHP and switch the same model to pairwise for the finalists without rebuilding it. Sensitivity analysis answers the Friday question directly, and group decision making lets each stakeholder rate separately so you can see where the disagreement actually sits.
If you want to try the pairwise mechanics on a small problem first, the free online AHP calculator runs in the browser with nothing to install.