Why can't simply adding more processors keep speeding up a simulation?

Amdahl's law shows that any serial portion of the work caps the speedup no matter how many processors are added, and communication overhead grows with processor count, so beyond some point extra processors yield diminishing or negative returns for a fixed problem size.

Are faster algorithms or faster hardware more important?

Both matter, but algorithmic improvements such as fast multipole and particle-mesh methods have often delivered larger gains than hardware alone, because they change how the cost scales with problem size rather than just the constant factor.

High-Performance Computing in Physics

Modern physics simulations outgrow any single processor, so high-performance computing harnesses thousands of cores, accelerators and clever algorithms to run the largest molecular, lattice and astrophysical calculations.

Definition

High-performance computing in physics is the use of parallel hardware, accelerators and scalable algorithms to perform physics simulations far larger or faster than serial computation allows, while managing communication, load balancing and numerical scalability.

Scope

This area covers the computational infrastructure of large-scale physics: parallel programming with distributed and shared memory, GPU and accelerator computing, and scalable algorithms such as fast N-body and particle-mesh methods. It emphasizes how physics problems are mapped onto parallel hardware and what limits their scaling.

Sub-topics

Core questions

How are physics simulations decomposed across many processors?
What does Amdahl's law say about the limits of parallel speedup?
How do GPUs accelerate the data-parallel kernels common in physics?
How do scalable algorithms reduce the cost of long-range interactions?

Key theories

Domain decomposition and message passing: Large simulations are split across processors by partitioning the physical domain, with processors exchanging boundary data by message passing, so scalability depends on balancing computation against communication.
Amdahl's law and scaling limits: The achievable speedup from parallelization is bounded by the fraction of work that remains serial, which sets fundamental limits on how many processors can be used effectively for a fixed problem.
Scalable algorithms: Fast multipole, tree and particle-mesh methods reduce the cost of long-range interactions from quadratic to near-linear in particle number, making large simulations feasible independently of raw hardware speed.

Clinical relevance

High-performance computing enables the largest molecular dynamics, lattice quantum chromodynamics, cosmological N-body and climate simulations, and the same techniques accelerate data analysis across experimental and observational physics.

History

Scientific computing drove parallel hardware from vector supercomputers through massively parallel clusters to today's GPU-accelerated machines; algorithmic advances such as the fast multipole method, named a top algorithm of the twentieth century, were as important as hardware in enabling large physics simulations.

Key figures

Gene Amdahl
Peter Pacheco
Leslie Greengard

Seminal works

amdahl1967
pacheco2011

Frequently asked questions

Why can't simply adding more processors keep speeding up a simulation?: Amdahl's law shows that any serial portion of the work caps the speedup no matter how many processors are added, and communication overhead grows with processor count, so beyond some point extra processors yield diminishing or negative returns for a fixed problem size.
Are faster algorithms or faster hardware more important?: Both matter, but algorithmic improvements such as fast multipole and particle-mesh methods have often delivered larger gains than hardware alone, because they change how the cost scales with problem size rather than just the constant factor.