No-U-Turn Sampler (NUTS)

Standard HMC requires the user to specify two parameters: the leapfrog step size ε and the number of leapfrog steps L. Choosing L poorly leads either to slow mixing (too small) or wasted computation when the trajectory doubles back (too large). NUTS replaces the fixed L with the No-U-Turn stopping criterion — it doubles the trajectory until it detects a U-turn and then selects a point from the resulting tree — so L is set automatically. Step size is handled by dual averaging during warm-up. The result is that NUTS reproduces the performance of well-tuned HMC without requiring the user to tune L at all.

A divergent transition occurs when the leapfrog integrator encounters a region of the posterior with very high curvature and the simulated Hamiltonian deviates substantially from its initial value. This signals that the sampler has explored a region it cannot integrate accurately, and the resulting samples from that region are biased. In Stan and PyMC, divergences are flagged explicitly. Even a small number of divergences (say, more than about 0.1% of post-warm-up iterations) warrants investigation. The remedy is typically reparameterisation of the model (e.g. switching from a centred to a non-centred parameterisation for hierarchical models) or increasing the target acceptance rate (which reduces the step size and improves integration accuracy).

Run at least four chains from dispersed starting values and check: (1) R-hat < 1.01 for all parameters — values above this indicate the chains have not converged to a common distribution; (2) bulk and tail effective sample size (ESS) — aim for at least 100 per chain (400 total) for reliable posterior summaries and at least 400 for stable tail quantiles; (3) the number of divergent transitions should be near zero after reparameterisation; (4) energy diagnostic (E-BFMI) should be above roughly 0.2; (5) trace plots should show good mixing with no trends or stuck regions.

No — NUTS requires gradients of the log-posterior with respect to the parameters, which are undefined for discrete quantities. Models with discrete parameters must either marginalize those parameters out analytically (as Stan requires) or handle them with a separate sampler. PyMC supports some compound samplers that pair NUTS for continuous parameters with Metropolis-Hastings steps for discrete ones, but the joint mixing can be slow when the discrete and continuous parameters are highly dependent.