Decision Theory and Utility
Decision theory combines probabilities of outcomes with a numerical utility over those outcomes to define and compute rational choice as the maximization of expected utility.
Definition
Decision theory studies how an agent with probabilistic beliefs and preferences encoded as utilities should choose among actions; the normative answer is to select the action with the highest expected utility.
Scope
This topic covers the foundations of decision making under uncertainty: utility theory and the axioms that justify representing preferences by a utility function, the principle of maximum expected utility, decision networks (influence diagrams) that combine chance, decision, and utility nodes, and the value of information that quantifies how much an observation is worth. It addresses how rational single decisions are framed and solved. Sequential decision making over time is treated under Markov decision processes, and strategic interaction among agents under multi-agent systems.
Core questions
- How can rational preferences be represented by a numerical utility function?
- Why should a rational agent maximize expected utility?
- How do decision networks (influence diagrams) represent and solve a decision problem?
- How is the value of acquiring additional information computed?
Key concepts
- utility function
- preferences and lotteries
- rationality axioms
- maximum expected utility
- decision networks (influence diagrams)
- chance, decision, and utility nodes
- value of information
- risk attitude
Key theories
- Expected utility theory
- Under a set of rationality axioms on preferences over uncertain prospects, there exists a utility function such that the preferred choice is always the one with the highest expected utility, giving a normative foundation for decision making under uncertainty.
- Decision networks (influence diagrams)
- Influence diagrams extend Bayesian networks with decision nodes and a utility node, providing a compact graphical representation of a decision problem whose optimal policy can be computed by probabilistic inference and expected-utility maximization.
- Value of information
- Information value theory quantifies how much an agent should be willing to pay to observe an uncertain quantity before deciding, by comparing expected utility with and without the observation, guiding when to gather more evidence.
Clinical relevance
Decision-theoretic methods support medical and clinical decision analysis, automated planning of information gathering, recommendation and pricing systems, and the design of rational autonomous agents, by making explicit how uncertain beliefs and preferences combine into justified choices.
History
Expected utility theory was axiomatized by von Neumann and Morgenstern (1944) and given a subjective-probability foundation by Savage (1954). Howard's information value theory (1966) and the later development of influence diagrams brought decision theory into AI as a practical framework for building rational decision-making agents.
Key figures
- John von Neumann
- Oskar Morgenstern
- Leonard J. Savage
- Ronald A. Howard
- Ross D. Shachter
Related topics
Seminal works
- vonneumann1944
- savage1954
- howard1966
Frequently asked questions
- What is the principle of maximum expected utility?
- It states that a rational agent facing uncertainty should choose the action whose probability-weighted average utility over possible outcomes is highest. Under standard axioms on preferences, this principle uniquely characterizes rational choice.
- What is the value of information?
- The value of information is how much an agent's expected utility would improve if it could observe some uncertain quantity before deciding. It tells a rational agent when gathering more evidence is worthwhile and when it would not change the best action enough to justify the cost.