Continuous-Time Markov Chains
A continuous-time Markov chain holds each state for an exponential time and then jumps to another, its dynamics governed by a generator matrix of transition rates rather than a single-step transition matrix.
Definition
A continuous-time Markov chain is a Markov process on a countable state space that remains in each state for an exponentially distributed time and then jumps according to fixed probabilities, with the holding rates and jump probabilities summarized in a generator matrix.
Scope
The topic covers the jump-and-hold construction with exponential holding times and an embedded jump chain, the generator or Q-matrix of transition rates, the Kolmogorov forward and backward differential equations for the transition probabilities, the matrix-exponential solution, explosion and regularity, birth-and-death processes, and the long-run behavior governed by stationary distributions.
Core questions
- How is a continuous-time chain built from exponential holding times and jump probabilities?
- What is the generator matrix, and how does it determine the transition probabilities?
- How do the Kolmogorov forward and backward equations describe the evolution in time?
- When can the chain make infinitely many jumps in finite time, and how is this excluded?
Key concepts
- generator matrix
- exponential holding times
- embedded jump chain
- Kolmogorov forward and backward equations
- birth-and-death process
Key theories
- Generator and the Kolmogorov equations
- The off-diagonal generator entries give jump rates and the diagonal the total exit rates; the transition-probability matrix solves the forward and backward differential equations driven by the generator, with the matrix exponential of the generator as its formal solution.
- Jump-chain and holding-time construction
- A continuous-time chain can be realized by an embedded discrete-time jump chain together with state-dependent exponential holding times, which separates where the process goes from how long it waits and makes simulation and analysis straightforward.
Clinical relevance
Continuous-time Markov chains model queueing and telecommunication networks, the kinetics of ion channels and chemical reaction networks, population and epidemic models in continuous time, and the rating-migration models of credit risk; their generator formulation connects directly to the differential equations used to compute transient and equilibrium behavior.
History
Kolmogorov derived the forward and backward differential equations for continuous-time transition probabilities in 1931, and Feller analyzed their solutions, explosion, and boundary behavior, establishing the generator-based theory that underlies modern treatments of jump Markov processes.
Key figures
- Andrey Kolmogorov
- William Feller
- Agner Krarup Erlang
Related topics
Seminal works
- norris1997
Frequently asked questions
- How does a continuous-time Markov chain differ from a discrete-time one?
- A discrete-time chain moves at fixed integer steps, while a continuous-time chain stays in each state for a random exponential time before jumping, so its dynamics are described by transition rates in a generator rather than by one-step transition probabilities.
- What is explosion in this context?
- Explosion is the possibility that the chain makes infinitely many jumps in a finite time interval, which can happen when holding rates grow without bound; a chain is called regular or non-explosive when this has probability zero.