Combining Experts’ Judgments: Comparison of Algorithmic Methods using Synthetic Data
Expert judgment (or expert elicitation) is a formal process for eliciting judgments from subject-matter experts about the value of a decision-relevant quantity. Judgments in the form of subjective probability distributions are obtained from several experts, raising the question how best to combine information from multiple experts. A number of algorithmic approaches have been proposed, of which the most commonly employed is the equal-weight combination (the average of the experts’ distributions). We evaluate the properties of five combination methods (equal-weight, best-expert, performance, frequentist, and copula) using simulated expert-judgment data for which we know the process generating the experts’ distributions. We examine cases in which two well-calibrated experts are of equal or unequal quality and their judgments are independent, positively or negatively dependent. In this setting, the copula, frequentist, and best-expert approaches perform better and the equal-weight combination method performs worse than the alternative approaches.
Year of publication: |
2012-03
|
---|---|
Authors: | Hammitt, James ; Zhang, Yifan |
Institutions: | Toulouse School of Economics (TSE) |
Saved in:
freely available
Saved in favorites
Similar items by person
-
Expert Elicitation of the Value per Statistical Life in an Air Pollution Context
Hammitt, James, (2012)
-
Risk and Choice: A Research Saga
Gollier, Christian, (2013)
-
The Intergenerational Transmission of Education: Evidence from Taiwanese Adoptions
Hammitt, James, (2011)
- More ...