Multi-Objective Dynamic Programming — Pareto-optimal policies over sequential decisions

Running DP separately for each objective yields individual single-objective optima, which are generally infeasible to achieve simultaneously and miss the trade-off structure entirely. MODP propagates vector-valued Pareto sets through the state space, capturing all non-dominated combinations of objective values achievable by any single policy — including solutions in non-convex regions of the Pareto front that weighted-sum approaches cannot find.

MODP becomes intractable when the state space has more than a handful of discrete variables, when the planning horizon is very long, or when there are more than three or four objectives (causing the Pareto front at each state to grow exponentially). Approximate methods such as multi-objective reinforcement learning or heuristic Pareto search are generally preferred in such cases.

Yes. In a stochastic setting, the vector value function stores expected Pareto-optimal reward vectors, computed using the probability-weighted sum over successor states. This extends naturally to multi-objective Markov decision processes (MOMDPs), though stochasticity further increases computational demands.

No — this is one of MODP's key advantages. Weights or preference information are not required during the optimization phase. The algorithm computes the full Pareto front, and the decision-maker applies their preferences afterward to select a specific policy.

MODP outputs a set of Pareto-optimal policies, each associated with a vector of cumulative objective values. No single policy dominates all others across every objective. The decision-maker inspects this front to identify policies that match their priority ordering, often aided by visualization tools such as scatter plots or parallel coordinate plots of the objective vectors.