Applied information economics (AIE) is a decision analysis method developed by Douglas W. Hubbard and partially described in his book "How to Measure Anything: Finding the Value of Intangibles in Business"[1]. It builds on several methods from decision theory and risk analysis including the use of Monte Carlo methods. However, unlike some other modeling approaches with simulations, AIE incorporates the following:
Calibrated probability assessment. This is a method for training estimators and experts (who are relied on for the inputs in Monte Carlo methods) to be neutrally confident about their assigned probabilities[2]. That is, their probabilities are neither over confident (too high) nor under confident (too low). A Monte Carlo model created with calibrated estimates is the best possible representation of the decision makers' current state of uncertainty.
Computing the value of additional information. AIE uses information value calculations from decision theory such as the expected value of perfect information and the value of imperfect (partial) information. Often, this is done for a large number of uncertain variables in some type of decision model or business case. The result will reveal where efforts to reduce uncertainty by making further measurements are best spent.
Empirical methods applied according to the information value of the measurement. This step is, in fact, the reason for the name of the method. Most Monte Carlo modeling experts stop modeling after the first (uncalibrated) probability estimates from experts and there is usually little emphasis on further measurements with empirical methods. Since AIE computes the value of additional information, measurement can be selective and focused. This step often results in a very different set of measurement priorities than would otherwise have been used.
Various optimization methods including Modern Portfolio Theory. MPT and other methods are applied to determine ideal risk and return positions for a set of alternatives.
Practitioners of AIE claim that if something has an impact on an organization, it must be observable and, therefore, measurable.
AIE differs in several ways from other popular methods of decision analysis:
Unlike the accounting-style business case or cost benefit analysis, it does not rely entirely on point estimates of uncertain values. Since it uses the Monte Carlo method, uncertainty can be modeled explicitly.
Unlike most decision-theory analysis of uncertain decisions, the AIE model starts with "calibrating" the estimators. Calibrated probability assessments appear to have the effect of reducing over-confidence and under-confidence of experts used for estimating the probabilities.
The calculation of the value of information is used to guide further measurement efforts. Some research shows that without such guidance decision makers tend to choose less relevant measures.
AIE converts all values to economic terms so that powerful financial and economic optimization methods can be employed. This is unlike some methods that rely mostly on subjective scores.
AIE does tend to be somewhat more elaborate than these alternatives. But practitioners argue that it is no more complicated than analysis methods used in many other fields, as long as trained specialists are used. It also becomes more important to choose rigor over simplicity when the decisions being analyzed are much larger and riskier.
Limitations
Until enough experts are fully trained in this method, its complexity might limit its adoption by managers accustomed to traditional cost-benefit analysis.
Like traditional cost-benefit analysis it does not guarantee that some important factor won't be excluded if nobody thinks of adding it. It simply ensures that the factors included are at least calibrated estimates (not overconfident or underconfident) and that further reduction of uncertainty (by measurement efforts) is optimized and applied to the right factors.
Some of the same limitations as Monte Carlo simulations intervene. For example, if some variables are (unknown to the analysts) covariant, then the Monte Carlo would generate a distribution of results that does not fit reality.
Calibration removes certain systemic human estimate biases, but not all of them. It is a significant improvement on Monte Carlos models that have no calibration for initial estimates, but there is no guarantee that other biases of experts will not be introduced into the model.
It uses MPT in a modified and limited manner. Since many business investment are evaluated as opportunities arise, and not in a large "batch", they are often evaluated one at a time. The only persistently used component of MPT is the investment boundary, modified for the assessment of single investments of various sizes.
Adoption of AIE
Developed and first used in 1995, AIE is relatively new compared to most decision making methods in business and government. However, as early as 1998, a reader survey conducted by InformationWeek in 1998 showed that 12% of respondants use AIE for IT metrics.[3]. Being a more quantitative method, the adoption may be more difficult than some methods. Yet, the theoretical rigour and practical successes have drawn strong proponents.
Very few people have sufficient training and experience in AIE to be able to conduct an analysis without supervision, but more continue to gain certification.
Encyclopedia
top
