Confidence Interval

A 95% CI is the range of values calculated from sample data such that if the sampling procedure were repeated many times, approximately 95% of the calculated intervals would contain the true population parameter. Critically, this does NOT mean there is a 95% probability the true value is in this specific interval (that is the Bayesian interpretation). The frequentist CI is either correct or incorrect; the 95% refers to the long-run behavior of the procedure.

95% is standard in most fields and balances precision with parsimony. Use 99% when higher confidence is needed (e.g., nuclear safety, expensive clinical trials) or 90% when resources are limited and you can tolerate slightly more uncertainty. The choice is context-dependent and should be pre-specified. 95% is conventional and defensible.

Likely yes, but not automatically. Visual overlap is a rough guide. For a precise test, the overlap must be considered relative to the critical values and confidence levels. A rule of thumb: if 95% CIs do not overlap, p < 0.05 (two-sided) is likely. However, always conduct a formal statistical test if the decision is critical.

In principle, yes. Standard formulas exist for means, proportions, medians, and differences between groups (normal or non-normal data). For complex parameters (e.g., custom effect sizes, multilevel model coefficients), use bootstrapping or Bayesian methods. Most statistical software can compute CIs via resampling if closed-form formulas are unavailable.

Not necessarily. A very narrow CI may result from a very large sample size, which makes even trivial effects appear precise. Instead, evaluate precision in the context of the effect size and practical significance. A narrow CI around a clinically insignificant effect is not as valuable as a wide CI around a large, meaningful effect.