How does a continuous-time Markov chain differ from a discrete-time one?

Transitions occur at random continuous times rather than at fixed steps; the chain holds each state for an exponential time and then jumps, with dynamics governed by transition rates rather than a one-step probability matrix.

What is the infinitesimal generator?

It is the matrix of transition rates whose off-diagonal entries give the rate of jumping between states and whose rows sum to zero; the transition probabilities over time are the matrix exponential of the generator times the elapsed time.

Continuous-Time Markov Chains

A continuous-time Markov chain moves between a discrete set of states at random times, holding each state for an exponentially distributed duration before jumping according to fixed transition rates.

Etsi aihe työkalulla PaperMindTulossaFind papers & topics

Tools & resources

Lataa diat

Learn & explore

VideoTulossa

Definition

A continuous-time Markov chain is a stochastic process on a countable state space indexed by continuous time whose future given the present is independent of the past, characterised by a generator matrix of transition rates so that holding times are exponential and jumps follow an embedded chain.

Scope

This area covers the holding-time and jump-chain description, the infinitesimal generator and transition rates, the Kolmogorov forward and backward differential equations, stationary distributions and reversibility, birth-death processes, and the construction of chains from their embedded discrete-time jump chains.

Sub-topics

Core questions

How do exponential holding times and jump probabilities define a continuous-time chain?
What is the generator matrix and how does it encode transition rates?
How do the Kolmogorov forward and backward equations describe the evolution of transition probabilities?
When does a continuous-time chain possess a stationary distribution?

Key theories

Generator and the Kolmogorov equations: The infinitesimal generator collects the instantaneous transition rates, and the transition probability matrix solves the forward and backward Kolmogorov differential equations, giving the time evolution as a matrix exponential of the generator.
Jump-chain and holding-time construction: A continuous-time chain is built from an embedded discrete-time jump chain that chooses successive states and independent exponential holding times whose rates depend on the current state, separating where the chain goes from when it moves.

Clinical relevance

Continuous-time Markov chains model queueing systems, chemical reaction networks, population dynamics, epidemic spread, and reliability of multi-component systems, providing tractable continuous-time descriptions whose equilibrium and transient behaviour can be computed from the generator.

History

Kolmogorov's 1931 paper on analytic methods in probability introduced the differential equations governing transition probabilities, and Feller's work in the 1930s and 1940s clarified the construction and explosion behaviour of continuous-time chains, establishing the generator-based theory used today.

Key figures

Andrey Kolmogorov
William Feller
Alfred Lotka

Seminal works

norris1997

Frequently asked questions

How does a continuous-time Markov chain differ from a discrete-time one?: Transitions occur at random continuous times rather than at fixed steps; the chain holds each state for an exponential time and then jumps, with dynamics governed by transition rates rather than a one-step probability matrix.
What is the infinitesimal generator?: It is the matrix of transition rates whose off-diagonal entries give the rate of jumping between states and whose rows sum to zero; the transition probabilities over time are the matrix exponential of the generator times the elapsed time.