Jump Processes and Embedded Chains
A continuous-time Markov chain can be decomposed into a discrete-time jump chain that records the sequence of states visited and exponential holding times that record how long each state is occupied.
Definition
The embedded chain of a continuous-time Markov chain is the discrete-time Markov chain of successively visited states, which together with independent exponential holding times whose rates depend on the current state fully determines the continuous-time process.
Scope
This topic covers the embedded jump chain and its transition probabilities, exponential holding times with state-dependent rates, the equivalence between the generator description and the jump-hold construction, explosion and the possibility of infinitely many jumps in finite time, and the use of uniformisation to relate continuous-time chains to discrete-time ones.
Core questions
- How is the embedded jump chain extracted from a continuous-time chain?
- Why are holding times exponentially distributed, and how do their rates depend on the state?
- When can the continuous-time chain explode by making infinitely many jumps in finite time?
- How does uniformisation convert a continuous-time chain into a discrete-time one?
Key theories
- Jump-hold construction
- Starting from a state, the chain waits an exponential time whose rate is the total exit rate and then jumps to a new state chosen by the embedded chain's transition probabilities, reconstructing the full continuous-time process from these two ingredients.
- Explosion and non-conservativeness
- If exit rates grow fast enough along a trajectory, the cumulative holding times can converge and the chain makes infinitely many jumps in finite time, an explosion that must be excluded for the transition semigroup to be honest.
Clinical relevance
The jump-hold construction is the basis of exact stochastic simulation of Markov chains, including the Gillespie algorithm for chemical reaction networks, and uniformisation gives a stable numerical method for computing transient distributions in reliability and performance models.
History
Feller and Doob established the jump-hold representation and the explosion phenomenon in the 1940s, clarifying when a continuous-time chain is uniquely determined by its rates; the construction later underpinned exact simulation methods such as Gillespie's 1976 algorithm for chemical kinetics.
Key figures
- William Feller
- Joseph Doob
- Daniel Gillespie
Related topics
Seminal works
- norris1997
Frequently asked questions
- What is the embedded chain of a continuous-time Markov chain?
- It is the discrete-time Markov chain that records only the sequence of distinct states the process visits, ignoring how long it stays in each, and it captures where the process goes.
- What is explosion?
- Explosion occurs when a continuous-time chain makes infinitely many jumps within a finite time interval because its holding times shrink too quickly; well-behaved chains are constructed to avoid it.