What is the difference between strong and weak scaling?

Strong scaling keeps the problem size fixed and asks how much faster more processors make it, which Amdahl's law limits. Weak scaling grows the problem with the processor count, which, as Gustafson noted, lets very large machines stay efficient for correspondingly large simulations.

Why does communication limit parallel performance?

Processors must exchange boundary and global data, and this communication takes time that does not shrink as fast as the computation when more processors are added, so beyond a point communication overhead dominates and efficiency falls.

Parallel Computing in Physics

Parallel computing splits a physics simulation across many processors that work simultaneously, and getting it right means decomposing the problem, coordinating data exchange and understanding the laws that bound speedup.

Definition

Parallel computing in physics is the practice of dividing a simulation into tasks that execute concurrently on multiple processors or cores, coordinating their data and synchronization to solve larger problems faster than serial execution.

Scope

This topic covers parallel programming for physics: shared-memory threading and distributed-memory message passing, domain decomposition and load balancing, communication patterns and overhead, and the scaling laws of Amdahl and Gustafson. It treats how physical models map onto parallel hardware and how scalability is analyzed.

Core questions

How is a physics simulation decomposed across processors with balanced load?
How do shared-memory and distributed-memory models differ in programming and scaling?
How does communication overhead limit parallel efficiency?
What do Amdahl's and Gustafson's laws predict about achievable speedup?

Key theories

Domain decomposition and load balancing: The simulation domain is partitioned among processors that compute their region and exchange boundary data; even distribution of work and minimal communication are what determine parallel efficiency.
Strong scaling and Amdahl's law: For a fixed problem, the serial fraction limits the maximum speedup, so adding processors gives diminishing returns, a constraint known as strong scaling and quantified by Amdahl's law.
Weak scaling and Gustafson's law: When the problem size grows with the number of processors, much larger speedups are attainable, as Gustafson observed, because the parallel workload expands while the serial part stays fixed.

Clinical relevance

Parallel computing makes large molecular dynamics, lattice field theory, fluid and astrophysical simulations possible by spreading them across clusters and supercomputers, and is a basic skill for any large-scale computational physics work.

History

Parallel scientific computing advanced from early vector and shared-memory machines to distributed clusters programmed with the message-passing interface; Amdahl's 1967 pessimism about speedup was complemented by Gustafson's 1988 insight that scaling up the problem changes the calculus.

Key figures

Gene Amdahl
John Gustafson
Peter Pacheco

Seminal works

amdahl1967
gustafson1988

Frequently asked questions

What is the difference between strong and weak scaling?: Strong scaling keeps the problem size fixed and asks how much faster more processors make it, which Amdahl's law limits. Weak scaling grows the problem with the processor count, which, as Gustafson noted, lets very large machines stay efficient for correspondingly large simulations.
Why does communication limit parallel performance?: Processors must exchange boundary and global data, and this communication takes time that does not shrink as fast as the computation when more processors are added, so beyond a point communication overhead dominates and efficiency falls.