The problem
Why side-by-side comparisons go wrong
Two job offers, two laptops, two vendors: most two-way choices are decided less by the facts than by which fact you saw first, which option was the default, and which differences are easy to see side by side. A fair comparison fixes the criteria and their importance before you look at the options, so the result is something you can check, not a feeling you defend.
The idea is old. In a 1772 letter to Joseph Priestley, Benjamin Franklin described his "moral or prudential algebra": list pros and cons in two columns, then strike out reasons of equal weight on each side until the balance is clear.
The method
How a weighted decision matrix works
A weighted decision matrix is the simplest multi-attribute model, the family Ralph Keeney and Howard Raiffa described in Decisions with Multiple Objectives (1976). Engineers use a relative, the Pugh matrix. The steps:
- Write the decision as one question. "Which offer should I accept?"
- List 3 to 6 independent criteria. "Salary" and "total pay" count the same thing twice. Phrase each so higher is better: "affordability", not "price".
- Weight each criterion from 1 to 5 before you score. 5 is a deal-breaker, 1 a tie-breaker.
- Score both options from 1 to 10, one criterion at a time, so the two scores are relative to each other.
- Multiply and add. Each total is the sum of weight times score.
- Test the result. If moving one weight by one point flips the winner, the options are close, not one better.
Free tool
Side-by-side weighted decision matrix
Name both options, pick the number of criteria, then set weights (1-5) and scores (1-10). Totals update as you type; nothing leaves this page.
Method: total = sum of weight × score; maximum = sum of weights × 10; margin = difference of totals, also as a share of the maximum. The flip check moves one weight by one point and reports any change that alters the winner. "Near tie" (under 5% of the maximum) is a rule of thumb, not a statistical test.
The traps
Five biases that tilt a side-by-side comparison
Anchoring
The first number you see pulls later estimates toward it. In Tversky and Kahneman's study (Science, 1974), a wheel landing on 10 or 65 moved median estimates of African countries in the UN to 25% and 45%. Write your budget down before you see a price.
The decoy effect
A third option that is clearly worse than one of the two shifts choice toward that one. Huber, Payne and Puto named it asymmetric dominance (Journal of Consumer Research, 1982). Compare the two real options alone.
Distinction bias
Side by side, small differences look large. Hsee and Zhang (Journal of Personality and Social Psychology, 2004) showed joint evaluation overweights gaps people barely notice in use. Would you notice it after a month?
Confirmation bias
Once you lean one way, you seek evidence that agrees. Nickerson's review (Review of General Psychology, 1998) traces it across many fields. Argue for the option you like less.
Status quo bias
The option you already have gets extra weight for being current (Samuelson and Zeckhauser, Journal of Risk and Uncertainty, 1988). Score it as if choosing it fresh.
Source: Dan Ariely, Predictably Irrational (2008). 100 MIT students saw web only $59, print only $125, print plus web $125: 16 chose web only, none chose print only, 84 chose print plus web. A second group saw only the two real options, and the split moved to 68/32. A classroom demonstration, not a large controlled study.
Checklist
A fair comparison in five checks
- Criteria and weights were written down before either option was scored.
- No two criteria measure the same benefit.
- Both options were scored with the same information.
- The winner survives a one-point change to any single weight.
- If the result is a near tie, the choice goes to the option that is cheaper to undo.
FAQ
Frequently asked questions
What is a weighted decision matrix?
A table of criteria, each with a weight for how much it matters. You score every option on every criterion and add up weight times score; the higher total wins on the criteria you chose.
How many criteria should I use?
Usually 3 to 6. Fewer rarely captures the trade-off; more tends to count the same benefit twice.
What if the two totals are very close?
Treat it as a tie. Look for a criterion you left out, or pick the option that is easier to reverse.
Should I set the weights before or after scoring?
Before. Weights set after scoring drift toward whatever your favourite option is good at.